[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:23 dist-test-slave-c910 krb5kdc[811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:23 dist-test-slave-c910 krb5kdc[811](info): set up 2 sockets
Jun 19 12:09:23 dist-test-slave-c910 krb5kdc[811](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 12:09:24.208108 804 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:24.333645 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:33379
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:33379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:24.413686 824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:24.413908 824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:24.413931 824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:24.415162 824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:24.415268 824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:24.415297 824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:24.415366 824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:24.415395 824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:24.417130 824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:33379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:33379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:24.417423 824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:24.417719 824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:24.423295 830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:24.423337 824 server_base.cc:1034] running on GCE node
W20260619 12:09:24.423416 829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:24.423610 832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:24.423941 824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:24.424196 824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:24.425339 824 hybrid_clock.cc:648] HybridClock initialized: now 1781870964425330 us; error 43 us; skew 500 ppm
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:24.427855 824 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:24.428359 824 webserver.cc:460] Webserver started at http://127.0.201.62:36679/ using document root <none> and password file <none>
I20260619 12:09:24.428589 824 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:24.428635 824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:24.428790 824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:24.429906 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "a94858f0c254493b8e2fc82db71c46fb"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "54a529ce90cf331d7987397badd06339"
server_key_iv: "5b301f212155be7745af8a70cf2f1a94"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.430290 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "a94858f0c254493b8e2fc82db71c46fb"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "54a529ce90cf331d7987397badd06339"
server_key_iv: "5b301f212155be7745af8a70cf2f1a94"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.431831 824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:24.432629 839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.432878 824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:24.432960 824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "a94858f0c254493b8e2fc82db71c46fb"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "54a529ce90cf331d7987397badd06339"
server_key_iv: "5b301f212155be7745af8a70cf2f1a94"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.433063 824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:24.458969 824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:24.461087 824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:24.461215 824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:24.464954 824 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:33379
I20260619 12:09:24.464986 891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:33379 every 8 connection(s)
I20260619 12:09:24.465344 824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:24.466295 892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:24.468503 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 824
I20260619 12:09:24.468588 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:24.468617 892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb: Bootstrap starting.
I20260619 12:09:24.469209 892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:24.469239 804 external_mini_cluster.cc:1428] Setting key 7e8f03e4bae5193753ad135187fa4913
I20260619 12:09:24.469504 892 log.cc:824] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:24.470319 892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb: No bootstrap required, opened a new log
I20260619 12:09:24.472055 892 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94858f0c254493b8e2fc82db71c46fb" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33379 } }
I20260619 12:09:24.472250 892 raft_consensus.cc:374] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:24.472288 892 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94858f0c254493b8e2fc82db71c46fb, State: Initialized, Role: FOLLOWER
I20260619 12:09:24.472414 892 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94858f0c254493b8e2fc82db71c46fb" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33379 } }
I20260619 12:09:24.472499 892 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:24.472529 892 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:24.472579 892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:24.473367 892 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94858f0c254493b8e2fc82db71c46fb" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33379 } }
I20260619 12:09:24.473454 892 leader_election.cc:302] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94858f0c254493b8e2fc82db71c46fb; no voters:
I20260619 12:09:24.473665 892 leader_election.cc:288] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:24.473733 897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:24.473855 897 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [term 1 LEADER]: Becoming Leader. State: Replica: a94858f0c254493b8e2fc82db71c46fb, State: Running, Role: LEADER
I20260619 12:09:24.473932 892 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:24.473940 897 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94858f0c254493b8e2fc82db71c46fb" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33379 } }
I20260619 12:09:24.474417 897 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94858f0c254493b8e2fc82db71c46fb. Latest consensus state: current_term: 1 leader_uuid: "a94858f0c254493b8e2fc82db71c46fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94858f0c254493b8e2fc82db71c46fb" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33379 } } }
I20260619 12:09:24.474509 897 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:24.474718 905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:24.475227 905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:24.475473 898 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a94858f0c254493b8e2fc82db71c46fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94858f0c254493b8e2fc82db71c46fb" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33379 } } }
I20260619 12:09:24.475718 898 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:24.476919 905 catalog_manager.cc:1347] Generated new cluster ID: 81210fe0eb304c11b8310e96eda3a687
I20260619 12:09:24.476970 905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:24.486421 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.470167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.0.201.62:33379)
0619 12:09:24.470355 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:24.470365 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:24.470611 (+ 246us) server_negotiation.cc:407] Connection header received
0619 12:09:24.472252 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:24.472260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:24.472313 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:24.472472 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:24.474284 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.475895 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.477420 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.478015 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.478128 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.482310 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:24.482315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:24.482315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:24.482325 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:24.484306 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.484785 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.484788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.484788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.484907 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.485187 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.485187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.485188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.485463 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:24.485637 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:24.485885 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 12:09:24.486010 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260619 12:09:24.492430 905 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:24.493105 905 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:24.499564 905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94858f0c254493b8e2fc82db71c46fb: Generated new TSK 0
I20260619 12:09:24.499725 905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:24.528149 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33379
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:24.610913 918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:24.611123 918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:24.611147 918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:24.612370 918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:24.612417 918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:24.612478 918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:24.613907 918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33379
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:24.614161 918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:24.614398 918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:24.619550 923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:24.619678 918 server_base.cc:1034] running on GCE node
W20260619 12:09:24.619567 926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:24.619603 924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:24.619992 918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:24.620186 918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:24.621356 918 hybrid_clock.cc:648] HybridClock initialized: now 1781870964621354 us; error 39 us; skew 500 ppm
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:24.623735 918 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:24.624150 918 webserver.cc:460] Webserver started at http://127.0.201.1:42115/ using document root <none> and password file <none>
I20260619 12:09:24.624389 918 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:24.624460 918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:24.624612 918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:24.625631 918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "b67b3d0868b343089d19b3405ac480c7"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "e4fa7b532210cd9b15c2056e956d3078"
server_key_iv: "52eabe0f113d830c1722c05208c5a37f"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.626000 918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "b67b3d0868b343089d19b3405ac480c7"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "e4fa7b532210cd9b15c2056e956d3078"
server_key_iv: "52eabe0f113d830c1722c05208c5a37f"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.627354 918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:09:24.628085 933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.628252 918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:24.628353 918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "b67b3d0868b343089d19b3405ac480c7"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "e4fa7b532210cd9b15c2056e956d3078"
server_key_iv: "52eabe0f113d830c1722c05208c5a37f"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.628433 918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:24.648478 918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:24.650429 918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:24.650604 918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:24.650858 918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:24.651168 918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:24.651252 918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.651288 918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:24.651314 918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.657560 918 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:43747
I20260619 12:09:24.657604 1046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:43747 every 8 connection(s)
I20260619 12:09:24.657956 918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:24.663475 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 918
I20260619 12:09:24.663568 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:24.663717 804 external_mini_cluster.cc:1428] Setting key ced05179083ae7b13fe82f44bf471a52
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:24.668218 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.658708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:55683 (local address 127.0.201.62:33379)
0619 12:09:24.658754 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:24.658756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:24.658904 (+ 148us) server_negotiation.cc:407] Connection header received
0619 12:09:24.659394 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:24.659395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:24.659406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:24.659468 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:24.660394 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.661265 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.662422 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.662634 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.662706 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.665018 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:24.665019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:24.665020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:24.665028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:24.666686 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.667090 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.667091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.667092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.667165 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.667445 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.667446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.667446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.667610 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:24.667689 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:24.668011 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 12:09:24.668094 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:09:24.668241 1049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.658625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33379 (local address 127.0.201.1:55683)
0619 12:09:24.658789 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:24.658830 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:24.659299 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:24.659547 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:24.659550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:24.660116 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:24.660304 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:24.660308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.661378 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.661379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:24.662330 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:24.662333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.662724 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.662725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:24.662822 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.663392 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:24.663402 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:24.664923 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:24.666788 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:24.666793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:24.666795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:24.667011 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:24.667259 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:24.667260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:24.667260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:24.667366 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:24.667682 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:24.667686 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:24.667866 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 12:09:24.667954 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:09:24.668070 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260619 12:09:24.668761 1047 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33379
I20260619 12:09:24.668879 1047 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:24.669047 1047 heartbeater.cc:511] Master 127.0.201.62:33379 requested a full tablet report, sending...
I20260619 12:09:24.669586 856 ts_manager.cc:194] Registered new tserver with Master: b67b3d0868b343089d19b3405ac480c7 (127.0.201.1:43747)
I20260619 12:09:24.670226 856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:55683
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:24.692284 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33379
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:24.773175 1052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:24.773375 1052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:24.773398 1052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:24.774545 1052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:24.774586 1052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:24.774732 1052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:24.776276 1052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33379
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:24.776533 1052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:24.776765 1052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:24.781802 1060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:24.781822 1058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:24.781978 1057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:24.782088 1052 server_base.cc:1034] running on GCE node
I20260619 12:09:24.782269 1052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:24.782449 1052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:24.783588 1052 hybrid_clock.cc:648] HybridClock initialized: now 1781870964783566 us; error 36 us; skew 500 ppm
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:24.786029 1052 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:24.786439 1052 webserver.cc:460] Webserver started at http://127.0.201.2:34415/ using document root <none> and password file <none>
I20260619 12:09:24.786679 1052 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:24.786751 1052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:24.786906 1052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:24.787964 1052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "dd4255a530c3415baa8e2573f97bb86e"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "f2205dc4816e673707cc901a6d43d92e"
server_key_iv: "f23c111ea41ebf4e949f2ad0dde347c1"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.788317 1052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "dd4255a530c3415baa8e2573f97bb86e"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "f2205dc4816e673707cc901a6d43d92e"
server_key_iv: "f23c111ea41ebf4e949f2ad0dde347c1"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.789583 1052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:24.790354 1067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.790539 1052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:24.790634 1052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "dd4255a530c3415baa8e2573f97bb86e"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "f2205dc4816e673707cc901a6d43d92e"
server_key_iv: "f23c111ea41ebf4e949f2ad0dde347c1"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.790712 1052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:24.806728 1052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:24.808739 1052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:24.808908 1052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:24.809120 1052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:24.809439 1052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:24.809502 1052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.809545 1052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:24.809572 1052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.815740 1052 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:46063
I20260619 12:09:24.815814 1180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:46063 every 8 connection(s)
I20260619 12:09:24.816130 1052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:24.817088 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1052
I20260619 12:09:24.817183 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:24.817350 804 external_mini_cluster.cc:1428] Setting key d80a77eeab444d1d2de6ba304769f304
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
I20260619 12:09:24.826781 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.816904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:41961 (local address 127.0.201.62:33379)
0619 12:09:24.816971 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:24.816973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:24.817202 (+ 229us) server_negotiation.cc:407] Connection header received
0619 12:09:24.817799 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:24.817800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:24.817810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:24.817859 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:24.818799 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.819567 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.821098 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.821290 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.821349 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.823875 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:24.823876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:24.823877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:24.823885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:24.825337 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.825830 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.825832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.825832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.825892 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.826141 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.826142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.826142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.826256 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:24.826333 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:24.826597 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 12:09:24.826680 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:09:24.826980 1183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.816896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33379 (local address 127.0.201.2:41961)
0619 12:09:24.817080 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:24.817115 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:24.817699 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:24.817940 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:24.817944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:24.818524 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:24.818702 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:24.818707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.819655 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.819656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:24.821006 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:24.821009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.821381 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.821383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:24.821538 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.822047 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:24.822057 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:24.823754 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:24.825440 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:24.825446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:24.825448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:24.825728 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:24.825959 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:24.825960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:24.825960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:24.826070 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:24.826324 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:24.826330 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:24.826470 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 12:09:24.826579 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:09:24.826752 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260619 12:09:24.827569 1181 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33379
I20260619 12:09:24.827680 1181 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:24.827867 1181 heartbeater.cc:511] Master 127.0.201.62:33379 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
I20260619 12:09:24.828274 856 ts_manager.cc:194] Registered new tserver with Master: dd4255a530c3415baa8e2573f97bb86e (127.0.201.2:46063)
I20260619 12:09:24.828814 856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:41961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:24.842465 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33379
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:24.925477 1186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:24.925769 1186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:24.925810 1186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:24.927887 1186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:24.927953 1186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:24.928006 1186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:24.930370 1186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33379
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:24.930650 1186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:24.930917 1186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:24.936733 1191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:24.937062 1192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:24.937182 1194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:24.937292 1186 server_base.cc:1034] running on GCE node
I20260619 12:09:24.937511 1186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:24.937767 1186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:24.938941 1186 hybrid_clock.cc:648] HybridClock initialized: now 1781870964938920 us; error 33 us; skew 500 ppm
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:24.942154 1186 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:24.942651 1186 webserver.cc:460] Webserver started at http://127.0.201.3:38231/ using document root <none> and password file <none>
I20260619 12:09:24.942914 1186 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:24.943005 1186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:24.943161 1186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:24.944542 1186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "87be0e83775f481a9ef214cb25c319ee"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "a54d371165d6f90ff87d37d6602e0e0f"
server_key_iv: "75ebb20635c35077ab66725d6f3eba97"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.944936 1186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "87be0e83775f481a9ef214cb25c319ee"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "a54d371165d6f90ff87d37d6602e0e0f"
server_key_iv: "75ebb20635c35077ab66725d6f3eba97"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.946492 1186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:24.947589 1201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.947784 1186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:24.947902 1186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "87be0e83775f481a9ef214cb25c319ee"
format_stamp: "Formatted at 2026-06-19 12:09:24 on dist-test-slave-c910"
server_key: "a54d371165d6f90ff87d37d6602e0e0f"
server_key_iv: "75ebb20635c35077ab66725d6f3eba97"
server_key_version: "encryptionkey@0"
I20260619 12:09:24.947974 1186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:24.970808 1186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:24.973469 1186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:24.973582 1186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:24.973763 1186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:24.974030 1186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:24.974059 1186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.974115 1186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:24.974138 1186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:24.980291 1186 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:39255
I20260619 12:09:24.980432 1314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:39255 every 8 connection(s)
I20260619 12:09:24.980698 1186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:09:24 dist-test-slave-c910 krb5kdc[811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:24.987802 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1186
I20260619 12:09:24.987890 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:24.988082 804 external_mini_cluster.cc:1428] Setting key 8f671d3b4ffcd325d2571dfc4a042425
I20260619 12:09:24.990041 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.981397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:45275 (local address 127.0.201.62:33379)
0619 12:09:24.981438 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:24.981440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:24.981675 (+ 235us) server_negotiation.cc:407] Connection header received
0619 12:09:24.982181 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:24.982182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:24.982195 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:24.982240 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:24.983169 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.983708 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.984715 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.984913 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.985000 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.987029 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:24.987030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:24.987030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:24.987037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:24.988846 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.989300 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.989301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.989301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.989367 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:24.989543 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:24.989543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:24.989544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:24.989679 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:24.989733 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:24.989918 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 12:09:24.989979 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:09:24.990280 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:24.981368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33379 (local address 127.0.201.3:45275)
0619 12:09:24.981490 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:24.981534 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:24.982053 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:24.982324 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:24.982328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:24.982873 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:24.983057 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:24.983061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.983815 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.983816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:24.984622 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:24.984624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:24.985008 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:24.985009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:24.985132 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:24.985622 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:24.985630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:24.986951 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:24.989016 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:24.989020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:24.989022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:24.989226 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:24.989420 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:24.989421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:24.989421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:24.989499 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:24.989725 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:24.989728 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:24.989832 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 12:09:24.989906 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:09:24.990016 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260619 12:09:24.990713 1315 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33379
I20260619 12:09:24.990820 1315 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:24.990990 1315 heartbeater.cc:511] Master 127.0.201.62:33379 requested a full tablet report, sending...
I20260619 12:09:24.991387 856 ts_manager.cc:194] Registered new tserver with Master: 87be0e83775f481a9ef214cb25c319ee (127.0.201.3:39255)
I20260619 12:09:24.991798 856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:45275
I20260619 12:09:25.000149 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:25.022373 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:25.014843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.0.201.1:43747)
0619 12:09:25.015005 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:25.015010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:25.015026 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:09:25.015052 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:25.015054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:25.015067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:25.015136 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:25.015740 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:25.016485 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:25.017190 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:25.017524 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:25.017588 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:25.019807 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:25.019814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:25.019815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:25.019821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:25.021471 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:25.021761 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:25.021764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:25.021764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:25.021830 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:25.021975 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:25.021975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:25.021975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:25.022085 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:25.022198 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:25.022233 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:09:25.022292 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
W20260619 12:09:25.022756 961 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60472
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:25.044780 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:25.037715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60478 (local address 127.0.201.1:43747)
0619 12:09:25.037795 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:25.037797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:25.037916 (+ 119us) server_negotiation.cc:407] Connection header received
0619 12:09:25.038012 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:25.038013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:25.038027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:25.038131 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:25.038693 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:25.039240 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:25.040025 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:25.040253 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:25.040341 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:25.042563 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:25.042565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:25.042565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:25.042573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:25.043826 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:25.044135 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:25.044136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:25.044136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:25.044186 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:25.044386 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:25.044387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:25.044387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:25.044485 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:25.044589 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:25.044651 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 12:09:25.044715 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:09:25.045351 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 918
I20260619 12:09:25.050238 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1052
I20260619 12:09:25.055349 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1186
I20260619 12:09:25.059595 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 824
2026-06-19T12:09:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1900 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[1329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[1329](info): set up 2 sockets
Jun 19 12:09:25 dist-test-slave-c910 krb5kdc[1329](info): commencing operation
krb5kdc: starting...
W20260619 12:09:26.102646 804 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:26 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:26.215643 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42805
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:39317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:26.295113 1342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:26.295351 1342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:26.295377 1342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:26.296541 1342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:26.296586 1342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:26.296603 1342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:26.296631 1342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:26.296645 1342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:26.298130 1342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:26.298390 1342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:26.298642 1342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:26.305531 1350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:26.305531 1348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:26.305660 1347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:26.305939 1342 server_base.cc:1034] running on GCE node
I20260619 12:09:26.306126 1342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:26.306414 1342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:26.307581 1342 hybrid_clock.cc:648] HybridClock initialized: now 1781870966307536 us; error 31 us; skew 500 ppm
Jun 19 12:09:26 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870966, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:26.310938 1342 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:26.311482 1342 webserver.cc:460] Webserver started at http://127.0.201.62:40185/ using document root <none> and password file <none>
I20260619 12:09:26.311761 1342 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:26.311842 1342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:26.311998 1342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:26.313087 1342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "066e520283c14a179527f2c89d4fd345"
format_stamp: "Formatted at 2026-06-19 12:09:26 on dist-test-slave-c910"
server_key: "7b77662a26c00b763fc3ee11dbd6985b"
server_key_iv: "bf797b73c1f5351c3512b2dd1af545e7"
server_key_version: "encryptionkey@0"
I20260619 12:09:26.313462 1342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "066e520283c14a179527f2c89d4fd345"
format_stamp: "Formatted at 2026-06-19 12:09:26 on dist-test-slave-c910"
server_key: "7b77662a26c00b763fc3ee11dbd6985b"
server_key_iv: "bf797b73c1f5351c3512b2dd1af545e7"
server_key_version: "encryptionkey@0"
I20260619 12:09:26.314782 1342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:26.315559 1357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:26.315764 1342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:26.315871 1342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "066e520283c14a179527f2c89d4fd345"
format_stamp: "Formatted at 2026-06-19 12:09:26 on dist-test-slave-c910"
server_key: "7b77662a26c00b763fc3ee11dbd6985b"
server_key_iv: "bf797b73c1f5351c3512b2dd1af545e7"
server_key_version: "encryptionkey@0"
I20260619 12:09:26.315959 1342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:26.325510 1342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:26.327399 1342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:26.327575 1342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:26.331324 1342 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42805
I20260619 12:09:26.331363 1409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42805 every 8 connection(s)
I20260619 12:09:26.331718 1342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:26.332553 1410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:26.334807 1410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345: Bootstrap starting.
I20260619 12:09:26.335445 1410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:26.335762 1410 log.cc:824] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:26.336447 1410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345: No bootstrap required, opened a new log
I20260619 12:09:26.338039 1410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066e520283c14a179527f2c89d4fd345" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42805 } }
I20260619 12:09:26.338171 1410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:26.338193 1410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066e520283c14a179527f2c89d4fd345, State: Initialized, Role: FOLLOWER
I20260619 12:09:26.338343 1410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066e520283c14a179527f2c89d4fd345" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42805 } }
I20260619 12:09:26.338421 1410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:26.338443 1410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:26.338515 1410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:26.339821 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1342
I20260619 12:09:26.339891 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:26.340104 804 external_mini_cluster.cc:1428] Setting key 515d4c000cea215c15e9c43bf1fcb271
I20260619 12:09:26.341284 1410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066e520283c14a179527f2c89d4fd345" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42805 } }
I20260619 12:09:26.341414 1410 leader_election.cc:302] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 066e520283c14a179527f2c89d4fd345; no voters:
I20260619 12:09:26.341583 1410 leader_election.cc:288] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:26.341609 1415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:26.341825 1415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [term 1 LEADER]: Becoming Leader. State: Replica: 066e520283c14a179527f2c89d4fd345, State: Running, Role: LEADER
I20260619 12:09:26.341853 1410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:26.342005 1415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066e520283c14a179527f2c89d4fd345" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42805 } }
I20260619 12:09:26.342432 1415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "066e520283c14a179527f2c89d4fd345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066e520283c14a179527f2c89d4fd345" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42805 } } }
I20260619 12:09:26.342593 1415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:26.342772 1415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 066e520283c14a179527f2c89d4fd345. Latest consensus state: current_term: 1 leader_uuid: "066e520283c14a179527f2c89d4fd345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "066e520283c14a179527f2c89d4fd345" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42805 } } }
I20260619 12:09:26.342881 1415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:26.343474 1424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:26.345858 1424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:26.347652 1424 catalog_manager.cc:1347] Generated new cluster ID: 5d4f7624f2c842f5a95f682743b9e43d
I20260619 12:09:26.347704 1424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:26 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:26.352440 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:26.340643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.0.201.62:42805)
0619 12:09:26.341176 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:26.341186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:26.341230 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:09:26.341345 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:26.341353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:26.341409 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:26.341546 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:26.342444 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:26.344077 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:26.344877 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:26.345577 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:26.345706 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:26.348510 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:26.348514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:26.348514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:26.348526 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:26.350466 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:26.350873 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:26.350876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:26.350876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:26.350984 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:26.351228 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:26.351229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:26.351229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:26.351523 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:26.352048 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:26.352112 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 12:09:26.352249 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":80,"threads_started":1}
I20260619 12:09:26.362701 1424 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:26.363528 1424 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:26.378088 1424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 066e520283c14a179527f2c89d4fd345: Generated new TSK 0
I20260619 12:09:26.378351 1424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
W20260619 12:09:27.771631 804 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.201.1: real 1.383s user 0.000s sys 0.003s
I20260619 12:09:27.771988 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42805
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:39317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:27.852154 1436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:27.852345 1436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:27.852367 1436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:27.853520 1436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:27.853564 1436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:27.853590 1436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:27.854984 1436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:27.855279 1436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:27.855512 1436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:27.860468 1442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:27.860611 1436 server_base.cc:1034] running on GCE node
W20260619 12:09:27.860532 1444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:27.860571 1441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:27.860936 1436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:27.861181 1436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:27.862346 1436 hybrid_clock.cc:648] HybridClock initialized: now 1781870967862323 us; error 37 us; skew 500 ppm
Jun 19 12:09:27 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:27.869007 1436 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:27.869431 1436 webserver.cc:460] Webserver started at http://127.0.201.1:42255/ using document root <none> and password file <none>
I20260619 12:09:27.869664 1436 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:27.869738 1436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:27.869886 1436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:27.871057 1436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "b189e5191c2a4098921653ab064cd02c"
format_stamp: "Formatted at 2026-06-19 12:09:27 on dist-test-slave-c910"
server_key: "8c8c53ed1032ab20554e0562d7241555"
server_key_iv: "6258bf56375d8e702bdae89302fdffff"
server_key_version: "encryptionkey@0"
I20260619 12:09:27.871469 1436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "b189e5191c2a4098921653ab064cd02c"
format_stamp: "Formatted at 2026-06-19 12:09:27 on dist-test-slave-c910"
server_key: "8c8c53ed1032ab20554e0562d7241555"
server_key_iv: "6258bf56375d8e702bdae89302fdffff"
server_key_version: "encryptionkey@0"
I20260619 12:09:27.873466 1436 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:27.874202 1451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:27.874432 1436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:27.874543 1436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "b189e5191c2a4098921653ab064cd02c"
format_stamp: "Formatted at 2026-06-19 12:09:27 on dist-test-slave-c910"
server_key: "8c8c53ed1032ab20554e0562d7241555"
server_key_iv: "6258bf56375d8e702bdae89302fdffff"
server_key_version: "encryptionkey@0"
I20260619 12:09:27.874626 1436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:27.893182 1436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:27.895594 1436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:27.895704 1436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:27.895890 1436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:27.896209 1436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:27.896243 1436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:27.896299 1436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:27.896325 1436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:27.901898 1436 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:38945
I20260619 12:09:27.901937 1564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:38945 every 8 connection(s)
I20260619 12:09:27.902261 1436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:27.906766 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1436
I20260619 12:09:27.906875 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:27.907050 804 external_mini_cluster.cc:1428] Setting key a6a679c73a18810a7f642f48fd0e3f7f
Jun 19 12:09:27 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:27.912842 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:27.902937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42805 (local address 127.0.201.1:36445)
0619 12:09:27.903089 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:27.903124 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:27.903642 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:27.903905 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:27.903908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:27.904449 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:27.904623 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:27.904627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:27.905956 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:27.905957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:27.906765 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:27.906767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:27.907258 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:27.907258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:27.907422 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:27.907933 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:27.907943 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:27.909478 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:27.911483 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:27.911489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:27.911492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:27.911747 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:27.911967 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:27.911968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:27.911968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:27.912062 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:27.912350 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:27.912353 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:27.912458 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 12:09:27.912564 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:09:27.912680 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260619 12:09:27.912861 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:27.902976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:36445 (local address 127.0.201.62:42805)
0619 12:09:27.903270 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:27.903272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:27.903283 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:27.903762 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:27.903764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:27.903778 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:27.903837 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:27.904732 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:27.905251 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:27.906858 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:27.907160 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:27.907249 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:27.909585 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:27.909587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:27.909588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:27.909595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:27.911391 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:27.911829 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:27.911830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:27.911830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:27.911896 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:27.912129 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:27.912130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:27.912131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:27.912282 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:27.912364 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:27.912616 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 12:09:27.912717 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":67,"threads_started":1}
I20260619 12:09:27.913364 1565 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42805
I20260619 12:09:27.913473 1565 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:27.913664 1565 heartbeater.cc:511] Master 127.0.201.62:42805 requested a full tablet report, sending...
I20260619 12:09:27.914204 1374 ts_manager.cc:194] Registered new tserver with Master: b189e5191c2a4098921653ab064cd02c (127.0.201.1:38945)
I20260619 12:09:27.915119 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:36445
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
W20260619 12:09:28.097468 804 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.201.2: real 0.190s user 0.000s sys 0.003s
I20260619 12:09:28.097844 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42805
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:39317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:28.180088 1571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:28.180290 1571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:28.180315 1571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:28.181378 1571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:28.181417 1571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:28.181445 1571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:28.182868 1571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:28.183045 1571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:28.183234 1571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:28.188050 1576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:28.188128 1571 server_base.cc:1034] running on GCE node
W20260619 12:09:28.188081 1577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:28.188050 1579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:28.188530 1571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:28.188807 1571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:28.189985 1571 hybrid_clock.cc:648] HybridClock initialized: now 1781870968189972 us; error 30 us; skew 500 ppm
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:28.192596 1571 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:28.193085 1571 webserver.cc:460] Webserver started at http://127.0.201.2:34241/ using document root <none> and password file <none>
I20260619 12:09:28.193315 1571 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:28.193387 1571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:28.193557 1571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:28.194595 1571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "3967c180913544ccb2995b935fd77ed1"
format_stamp: "Formatted at 2026-06-19 12:09:28 on dist-test-slave-c910"
server_key: "ec06c1607c8c76f1a81b02d80e301f65"
server_key_iv: "5f30649921449841d46fdef3a7568e0d"
server_key_version: "encryptionkey@0"
I20260619 12:09:28.195005 1571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "3967c180913544ccb2995b935fd77ed1"
format_stamp: "Formatted at 2026-06-19 12:09:28 on dist-test-slave-c910"
server_key: "ec06c1607c8c76f1a81b02d80e301f65"
server_key_iv: "5f30649921449841d46fdef3a7568e0d"
server_key_version: "encryptionkey@0"
I20260619 12:09:28.196461 1571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:28.197211 1586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:28.197382 1571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:28.197485 1571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "3967c180913544ccb2995b935fd77ed1"
format_stamp: "Formatted at 2026-06-19 12:09:28 on dist-test-slave-c910"
server_key: "ec06c1607c8c76f1a81b02d80e301f65"
server_key_iv: "5f30649921449841d46fdef3a7568e0d"
server_key_version: "encryptionkey@0"
I20260619 12:09:28.197597 1571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:28.222669 1571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:28.225085 1571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:28.225255 1571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:28.225503 1571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:28.225867 1571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:28.225924 1571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:28.225965 1571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:28.225993 1571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:28.231706 1571 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:43643
I20260619 12:09:28.231757 1699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:43643 every 8 connection(s)
I20260619 12:09:28.232120 1571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:28.232594 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1571
I20260619 12:09:28.232697 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:28.232870 804 external_mini_cluster.cc:1428] Setting key c62ceb4a56a65cdb823128f2241a354f
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
I20260619 12:09:28.243449 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.232880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:55595 (local address 127.0.201.62:42805)
0619 12:09:28.233055 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.233058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.233275 (+ 217us) server_negotiation.cc:407] Connection header received
0619 12:09:28.233947 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.233948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.233958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.233997 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:28.235315 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.236106 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.237649 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.237837 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.237891 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.240257 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:28.240258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:28.240259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:28.240266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:28.241842 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.242340 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.242341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.242341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.242403 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.242647 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.242648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.242648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.242769 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:28.242823 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.243308 (+ 485us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.243378 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260619 12:09:28.243667 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.232861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42805 (local address 127.0.201.2:55595)
0619 12:09:28.233116 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.233197 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.233845 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.234066 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.234070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.234866 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:28.235144 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.235150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.236228 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.236229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.237567 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.237570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.237922 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.237923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.238063 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.238550 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:28.238560 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:28.240162 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:28.241954 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:28.241960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:28.241962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:28.242241 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:28.242465 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:28.242466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:28.242466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:28.242588 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:28.242913 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:28.242919 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:28.243120 (+ 201us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.243294 (+ 174us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.243426 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":80,"threads_started":1}
I20260619 12:09:28.244203 1700 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42805
I20260619 12:09:28.244311 1700 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:28.244531 1700 heartbeater.cc:511] Master 127.0.201.62:42805 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
I20260619 12:09:28.244872 1374 ts_manager.cc:194] Registered new tserver with Master: 3967c180913544ccb2995b935fd77ed1 (127.0.201.2:43643)
I20260619 12:09:28.245234 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:55595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:28.258453 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42805
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:39317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:28.338166 1705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:28.338343 1705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:28.338366 1705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:28.339622 1705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:28.339668 1705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:28.339695 1705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:28.341080 1705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:28.341320 1705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:28.341547 1705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:28.346632 1711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:28.346679 1713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:28.346803 1705 server_base.cc:1034] running on GCE node
W20260619 12:09:28.346894 1710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:28.347095 1705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:28.347399 1705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:28.348552 1705 hybrid_clock.cc:648] HybridClock initialized: now 1781870968348544 us; error 25 us; skew 500 ppm
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:28.350890 1705 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:28.351402 1705 webserver.cc:460] Webserver started at http://127.0.201.3:45717/ using document root <none> and password file <none>
I20260619 12:09:28.351639 1705 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:28.351712 1705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:28.351855 1705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:28.352851 1705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "e68c07499eb341e3bc91ec0f97aa4115"
format_stamp: "Formatted at 2026-06-19 12:09:28 on dist-test-slave-c910"
server_key: "63b655303bd4d9a1ab6e702c451788dd"
server_key_iv: "2a40ccc21cf895878544dada45d45160"
server_key_version: "encryptionkey@0"
I20260619 12:09:28.353233 1705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "e68c07499eb341e3bc91ec0f97aa4115"
format_stamp: "Formatted at 2026-06-19 12:09:28 on dist-test-slave-c910"
server_key: "63b655303bd4d9a1ab6e702c451788dd"
server_key_iv: "2a40ccc21cf895878544dada45d45160"
server_key_version: "encryptionkey@0"
I20260619 12:09:28.354504 1705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:09:28.355307 1720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:28.355495 1705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:28.355589 1705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "e68c07499eb341e3bc91ec0f97aa4115"
format_stamp: "Formatted at 2026-06-19 12:09:28 on dist-test-slave-c910"
server_key: "63b655303bd4d9a1ab6e702c451788dd"
server_key_iv: "2a40ccc21cf895878544dada45d45160"
server_key_version: "encryptionkey@0"
I20260619 12:09:28.355674 1705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:28.372359 1705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:28.374641 1705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:28.374807 1705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:28.375021 1705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:28.375383 1705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:28.375437 1705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:28.375469 1705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:28.375497 1705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:28.381620 1705 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:34179
I20260619 12:09:28.381652 1833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:34179 every 8 connection(s)
I20260619 12:09:28.381968 1705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:28.382858 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1705
I20260619 12:09:28.382956 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:28.383127 804 external_mini_cluster.cc:1428] Setting key 499c7f1a11fef38b81445a066f3da2f7
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:28.391806 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.382801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:34081 (local address 127.0.201.62:42805)
0619 12:09:28.382848 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.382850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.383038 (+ 188us) server_negotiation.cc:407] Connection header received
0619 12:09:28.383795 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.383797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.383809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.383870 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:28.384875 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.385378 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.386359 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.386576 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.386632 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.388824 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:28.388826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:28.388826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:28.388833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:28.390364 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.390802 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.390803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.390803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.390861 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.391140 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.391141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.391141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.391334 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:28.391408 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.391681 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.391744 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:28.391970 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.382748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42805 (local address 127.0.201.3:34081)
0619 12:09:28.382909 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.382955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.383644 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.383942 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.383946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.384583 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:28.384767 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.384771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.385463 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.385464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.386278 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.386280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.386658 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.386660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.386757 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.387270 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:28.387278 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:28.388731 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:28.390464 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:28.390469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:28.390471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:28.390724 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:28.390934 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:28.390935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:28.390935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:28.391044 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:28.391415 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:28.391420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:28.391557 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.391648 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.391758 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260619 12:09:28.392575 1834 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42805
I20260619 12:09:28.392684 1834 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:28.392855 1834 heartbeater.cc:511] Master 127.0.201.62:42805 requested a full tablet report, sending...
I20260619 12:09:28.393276 1374 ts_manager.cc:194] Registered new tserver with Master: e68c07499eb341e3bc91ec0f97aa4115 (127.0.201.3:34179)
I20260619 12:09:28.393641 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:34081
I20260619 12:09:28.394963 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:09:28.402144 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.396510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.0.201.62:42805)
0619 12:09:28.396570 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.396572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.396697 (+ 125us) server_negotiation.cc:407] Connection header received
0619 12:09:28.396732 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.396732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.396741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.396780 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:28.397290 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.397747 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.398417 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.398597 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.398646 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.399711 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:28.399712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:28.399712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:28.399718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:28.401191 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.401518 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.401519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.401519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.401567 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.401735 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.401735 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.401736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.401861 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:28.401936 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.402038 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.402089 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:09:28.404405 1374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:28.405954 1374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:28.412539 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.408659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.0.201.3:34179)
0619 12:09:28.408889 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.408893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.408902 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:28.408939 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.408941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.408958 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.409052 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:28.409197 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.410117 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.411418 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.412115 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.412213 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.412271 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.412423 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.412469 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":89,"threads_started":1}
I20260619 12:09:28.412529 1848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.408395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:34179 (local address 127.0.0.1:44366)
0619 12:09:28.408580 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.408619 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.408684 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.409065 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.409068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.409070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:28.409134 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.409136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.410242 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.410243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.411327 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.411329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.412216 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.412217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.412295 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.412308 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.412407 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.412459 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260619 12:09:28.413049 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.408396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:43643 (local address 127.0.0.1:54836)
0619 12:09:28.408855 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.408865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.408919 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.409404 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.409406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.409408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:28.409452 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.409454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.410782 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.410782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.411858 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.411860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.412814 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.412815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.412874 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.412877 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.412928 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.412966 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260619 12:09:28.413411 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.408423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38945 (local address 127.0.0.1:54024)
0619 12:09:28.408993 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.409002 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.409043 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.409412 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.409412 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.409414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:28.409460 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.409461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.411276 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.411277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.412215 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.412217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.413209 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.413210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.413266 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.413270 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.413327 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.413366 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":5,"thread_start_us":91,"threads_started":1}
I20260619 12:09:28.413506 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.408677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.0.201.1:38945)
0619 12:09:28.408812 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.408818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.409100 (+ 282us) server_negotiation.cc:407] Connection header received
0619 12:09:28.409144 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.409147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.409163 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.409256 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:28.409543 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.410670 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.412311 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.413007 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.413068 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.413145 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.413373 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.413421 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260619 12:09:28.413481 1768 tablet_service.cc:1467] Processing CreateTablet for tablet 8563ef80cdc54d62a2f8befcb1edaf1d (DEFAULT_TABLE table=test-table [id=d4d42c4446f94e76b8951af1bf50bc77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:28.413847 1768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8563ef80cdc54d62a2f8befcb1edaf1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:28.414084 1499 tablet_service.cc:1467] Processing CreateTablet for tablet 8563ef80cdc54d62a2f8befcb1edaf1d (DEFAULT_TABLE table=test-table [id=d4d42c4446f94e76b8951af1bf50bc77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:28.414389 1499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8563ef80cdc54d62a2f8befcb1edaf1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:28.415385 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.408841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.0.201.2:43643)
0619 12:09:28.408991 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.408995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.409006 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:28.409048 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.409050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.409064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.409312 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:28.409557 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.410665 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.411954 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.412666 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.412733 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.412780 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.412964 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.413014 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260619 12:09:28.415988 1634 tablet_service.cc:1467] Processing CreateTablet for tablet 8563ef80cdc54d62a2f8befcb1edaf1d (DEFAULT_TABLE table=test-table [id=d4d42c4446f94e76b8951af1bf50bc77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:28.416289 1634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8563ef80cdc54d62a2f8befcb1edaf1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:28.417284 1852 tablet_bootstrap.cc:492] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: Bootstrap starting.
I20260619 12:09:28.417573 1853 tablet_bootstrap.cc:492] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: Bootstrap starting.
I20260619 12:09:28.418056 1852 tablet_bootstrap.cc:654] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:28.418174 1853 tablet_bootstrap.cc:654] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:28.418507 1853 log.cc:824] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:28.418663 1852 log.cc:824] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:28.419334 1853 tablet_bootstrap.cc:492] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: No bootstrap required, opened a new log
I20260619 12:09:28.419416 1853 ts_tablet_manager.cc:1397] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:28.419591 1856 tablet_bootstrap.cc:492] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: Bootstrap starting.
I20260619 12:09:28.420111 1856 tablet_bootstrap.cc:654] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:28.420593 1856 log.cc:824] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:28.421226 1856 tablet_bootstrap.cc:492] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: No bootstrap required, opened a new log
I20260619 12:09:28.421307 1856 ts_tablet_manager.cc:1397] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:28.421458 1853 raft_consensus.cc:348] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.421589 1853 raft_consensus.cc:374] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:28.421610 1853 raft_consensus.cc:729] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b189e5191c2a4098921653ab064cd02c, State: Initialized, Role: FOLLOWER
I20260619 12:09:28.421741 1853 consensus_queue.cc:260] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.421999 1853 ts_tablet_manager.cc:1428] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:09:28.422056 1565 heartbeater.cc:503] Master 127.0.201.62:42805 was elected leader, sending a full tablet report...
I20260619 12:09:28.422688 1852 tablet_bootstrap.cc:492] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: No bootstrap required, opened a new log
I20260619 12:09:28.422766 1852 ts_tablet_manager.cc:1397] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 12:09:28.423327 1856 raft_consensus.cc:348] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.423497 1856 raft_consensus.cc:374] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:28.423532 1856 raft_consensus.cc:729] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3967c180913544ccb2995b935fd77ed1, State: Initialized, Role: FOLLOWER
I20260619 12:09:28.423646 1856 consensus_queue.cc:260] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.423902 1856 ts_tablet_manager.cc:1428] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:09:28.423926 1700 heartbeater.cc:503] Master 127.0.201.62:42805 was elected leader, sending a full tablet report...
I20260619 12:09:28.424795 1852 raft_consensus.cc:348] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.424934 1852 raft_consensus.cc:374] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:28.424957 1852 raft_consensus.cc:729] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68c07499eb341e3bc91ec0f97aa4115, State: Initialized, Role: FOLLOWER
I20260619 12:09:28.425069 1852 consensus_queue.cc:260] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.425362 1852 ts_tablet_manager.cc:1428] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:09:28.425375 1834 heartbeater.cc:503] Master 127.0.201.62:42805 was elected leader, sending a full tablet report...
W20260619 12:09:28.482676 1701 tablet.cc:2369] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:09:28.632534 1835 tablet.cc:2369] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:28.649808 1860 raft_consensus.cc:482] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:28.649926 1860 raft_consensus.cc:504] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.650362 1860 leader_election.cc:288] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b189e5191c2a4098921653ab064cd02c (127.0.201.1:38945), 3967c180913544ccb2995b935fd77ed1 (127.0.201.2:43643)
W20260619 12:09:28.653270 1566 tablet.cc:2369] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:28.653430 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.650575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:43643 (local address 127.0.201.3:35903)
0619 12:09:28.650766 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.650776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.650842 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.651023 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.651023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.651025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:28.651089 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.651091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.651771 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.651772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.652538 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.652540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.653189 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.653190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.653260 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.653265 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.653314 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.653353 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":112,"threads_started":1}
I20260619 12:09:28.653666 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.650699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:35903 (local address 127.0.201.2:43643)
0619 12:09:28.650741 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.650742 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.650846 (+ 104us) server_negotiation.cc:407] Connection header received
0619 12:09:28.650908 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.650909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.650922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.650989 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:28.651161 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.651693 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.652625 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.653055 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.653530 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.653541 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.653559 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.653604 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:28.653702 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.650558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38945 (local address 127.0.201.3:56265)
0619 12:09:28.650605 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:28.650617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:28.650669 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:28.650862 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:28.650863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:28.650865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:28.650939 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.650941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.651651 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.651652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.652436 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:28.652438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.653013 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.653014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:28.653544 (+ 530us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.653549 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:28.653602 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:09:28.653635 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1}
I20260619 12:09:28.653744 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.650632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:56265 (local address 127.0.201.1:38945)
0619 12:09:28.650692 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.650695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.650706 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:28.650735 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.650736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.650749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.650821 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:28.651023 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.651572 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.652538 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.652926 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.652980 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.652988 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.653630 (+ 642us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.653682 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:09:28.653996 1654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8563ef80cdc54d62a2f8befcb1edaf1d" candidate_uuid: "e68c07499eb341e3bc91ec0f97aa4115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3967c180913544ccb2995b935fd77ed1" is_pre_election: true
I20260619 12:09:28.654024 1519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8563ef80cdc54d62a2f8befcb1edaf1d" candidate_uuid: "e68c07499eb341e3bc91ec0f97aa4115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b189e5191c2a4098921653ab064cd02c" is_pre_election: true
I20260619 12:09:28.654148 1654 raft_consensus.cc:2454] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e68c07499eb341e3bc91ec0f97aa4115 in term 0.
I20260619 12:09:28.654147 1519 raft_consensus.cc:2454] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e68c07499eb341e3bc91ec0f97aa4115 in term 0.
I20260619 12:09:28.654386 1722 leader_election.cc:302] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3967c180913544ccb2995b935fd77ed1, e68c07499eb341e3bc91ec0f97aa4115; no voters:
I20260619 12:09:28.654543 1860 raft_consensus.cc:2781] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:28.654600 1860 raft_consensus.cc:482] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:28.654623 1860 raft_consensus.cc:3037] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:28.655398 1860 raft_consensus.cc:504] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.655536 1860 leader_election.cc:288] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [CANDIDATE]: Term 1 election: Requested vote from peers b189e5191c2a4098921653ab064cd02c (127.0.201.1:38945), 3967c180913544ccb2995b935fd77ed1 (127.0.201.2:43643)
I20260619 12:09:28.655699 1654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8563ef80cdc54d62a2f8befcb1edaf1d" candidate_uuid: "e68c07499eb341e3bc91ec0f97aa4115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3967c180913544ccb2995b935fd77ed1"
I20260619 12:09:28.655743 1519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8563ef80cdc54d62a2f8befcb1edaf1d" candidate_uuid: "e68c07499eb341e3bc91ec0f97aa4115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b189e5191c2a4098921653ab064cd02c"
I20260619 12:09:28.655802 1519 raft_consensus.cc:3037] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:28.655807 1654 raft_consensus.cc:3037] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:28.656510 1519 raft_consensus.cc:2454] T 8563ef80cdc54d62a2f8befcb1edaf1d P b189e5191c2a4098921653ab064cd02c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e68c07499eb341e3bc91ec0f97aa4115 in term 1.
I20260619 12:09:28.656522 1654 raft_consensus.cc:2454] T 8563ef80cdc54d62a2f8befcb1edaf1d P 3967c180913544ccb2995b935fd77ed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e68c07499eb341e3bc91ec0f97aa4115 in term 1.
I20260619 12:09:28.656683 1722 leader_election.cc:302] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3967c180913544ccb2995b935fd77ed1, e68c07499eb341e3bc91ec0f97aa4115; no voters:
I20260619 12:09:28.656805 1860 raft_consensus.cc:2781] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:28.656975 1860 raft_consensus.cc:686] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [term 1 LEADER]: Becoming Leader. State: Replica: e68c07499eb341e3bc91ec0f97aa4115, State: Running, Role: LEADER
I20260619 12:09:28.657051 1860 consensus_queue.cc:237] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } }
I20260619 12:09:28.657982 1373 catalog_manager.cc:5495] T 8563ef80cdc54d62a2f8befcb1edaf1d P e68c07499eb341e3bc91ec0f97aa4115 reported cstate change: term changed from 0 to 1, leader changed from <none> to e68c07499eb341e3bc91ec0f97aa4115 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "e68c07499eb341e3bc91ec0f97aa4115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b189e5191c2a4098921653ab064cd02c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3967c180913544ccb2995b935fd77ed1" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e68c07499eb341e3bc91ec0f97aa4115" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 34179 } health_report { overall_health: HEALTHY } } }
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:28.699254 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.691355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.0.201.1:38945)
0619 12:09:28.691419 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.691422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.691549 (+ 127us) server_negotiation.cc:407] Connection header received
0619 12:09:28.691620 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.691621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.691633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.691679 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:28.692295 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.692806 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.693553 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.693753 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.693830 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.696294 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:28.696307 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:28.696307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:28.696322 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:28.697867 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.698280 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.698284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.698285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.698424 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.698665 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.698666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.698667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.698819 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:28.698991 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.699028 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.699110 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:28.721719 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.714731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.0.201.1:38945)
0619 12:09:28.714802 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.714804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.714892 (+ 88us) server_negotiation.cc:407] Connection header received
0619 12:09:28.714982 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.714983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.714993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.715070 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:28.715743 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.716252 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.717025 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.717230 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.717321 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.719606 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:28.719608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:28.719608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:28.719616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:28.720772 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.721070 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.721071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.721071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.721135 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.721296 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.721296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.721296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.721378 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:28.721457 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.721587 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.721654 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:28.743963 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:28.736235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.0.201.1:38945)
0619 12:09:28.736320 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:28.736322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:28.736429 (+ 107us) server_negotiation.cc:407] Connection header received
0619 12:09:28.736514 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:28.736515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:28.736527 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:28.736616 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:28.737224 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.737708 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.738500 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:28.738692 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:28.738785 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:28.741103 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:28.741105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:28.741105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:28.741111 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:28.742612 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.742939 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.742940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.742940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.742986 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:28.743324 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:28.743326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:28.743327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:28.743455 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:28.743577 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:28.743782 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 12:09:28.743858 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 12:09:28.744465 1479 tablet_service.cc:2908] Rejecting scan request for tablet 8563ef80cdc54d62a2f8befcb1edaf1d: Uninitialized: clean time has not yet been initialized
I20260619 12:09:28.745262 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1436
I20260619 12:09:28.750578 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1571
I20260619 12:09:28.755537 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1705
I20260619 12:09:28.759626 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1342
2026-06-19T12:09:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3700 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1876](info): set up 2 sockets
Jun 19 12:09:28 dist-test-slave-c910 krb5kdc[1876](info): commencing operation
krb5kdc: starting...
W20260619 12:09:29.794888 804 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:29 dist-test-slave-c910 krb5kdc[1876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:29.906512 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42601
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:29.986366 1889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:29.986624 1889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:29.986662 1889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:29.987896 1889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:29.988006 1889 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:09:29.988036 1889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:29.988067 1889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:29.988088 1889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:29.988111 1889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:29.989638 1889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42601
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:29.989919 1889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:29.990154 1889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:29.995107 1894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:29.995056 1895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:29.995170 1897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:29.995440 1889 server_base.cc:1034] running on GCE node
I20260619 12:09:29.995675 1889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:29.995924 1889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:29.997119 1889 hybrid_clock.cc:648] HybridClock initialized: now 1781870969997091 us; error 40 us; skew 500 ppm
Jun 19 12:09:29 dist-test-slave-c910 krb5kdc[1876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:29.999570 1889 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:30.000085 1889 webserver.cc:460] Webserver started at http://127.0.201.62:43867/ using document root <none> and password file <none>
I20260619 12:09:30.000317 1889 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:30.000391 1889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:30.000552 1889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:30.001601 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "91c3b80995944fca9b42178963cb8dc4"
format_stamp: "Formatted at 2026-06-19 12:09:29 on dist-test-slave-c910"
server_key: "315b806f41c9aa082587a06474e42010"
server_key_iv: "41e5b9c4b60b759b3b4c4c18b273b4ae"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.001964 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "91c3b80995944fca9b42178963cb8dc4"
format_stamp: "Formatted at 2026-06-19 12:09:29 on dist-test-slave-c910"
server_key: "315b806f41c9aa082587a06474e42010"
server_key_iv: "41e5b9c4b60b759b3b4c4c18b273b4ae"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.003302 1889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:09:30.004024 1904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.004196 1889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:30.004297 1889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "91c3b80995944fca9b42178963cb8dc4"
format_stamp: "Formatted at 2026-06-19 12:09:29 on dist-test-slave-c910"
server_key: "315b806f41c9aa082587a06474e42010"
server_key_iv: "41e5b9c4b60b759b3b4c4c18b273b4ae"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.004384 1889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:30.020995 1889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:30.022822 1889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:30.022998 1889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:30.027870 1889 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42601
I20260619 12:09:30.027922 1966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42601 every 8 connection(s)
I20260619 12:09:30.028296 1889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:30.028946 1967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.031132 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1889
I20260619 12:09:30.031167 1967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: Bootstrap starting.
I20260619 12:09:30.031250 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:30.031399 804 external_mini_cluster.cc:1428] Setting key 1b71aa456be380220fad8a4e5ece0a3a
I20260619 12:09:30.032013 1967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.032364 1967 log.cc:824] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:30.033103 1967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: No bootstrap required, opened a new log
I20260619 12:09:30.035461 1967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3b80995944fca9b42178963cb8dc4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42601 } }
I20260619 12:09:30.035599 1967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.035624 1967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c3b80995944fca9b42178963cb8dc4, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.035756 1967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3b80995944fca9b42178963cb8dc4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42601 } }
I20260619 12:09:30.035836 1967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:30.035861 1967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:30.035925 1967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:30.036439 1967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3b80995944fca9b42178963cb8dc4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42601 } }
I20260619 12:09:30.036559 1967 leader_election.cc:302] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c3b80995944fca9b42178963cb8dc4; no voters:
I20260619 12:09:30.036741 1967 leader_election.cc:288] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:30.036763 1972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:30.036875 1972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [term 1 LEADER]: Becoming Leader. State: Replica: 91c3b80995944fca9b42178963cb8dc4, State: Running, Role: LEADER
I20260619 12:09:30.036967 1972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3b80995944fca9b42178963cb8dc4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42601 } }
I20260619 12:09:30.037030 1967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:30.037271 1974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c3b80995944fca9b42178963cb8dc4. Latest consensus state: current_term: 1 leader_uuid: "91c3b80995944fca9b42178963cb8dc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3b80995944fca9b42178963cb8dc4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42601 } } }
I20260619 12:09:30.037348 1974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:30.037570 1976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:30.038075 1976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:30.038831 1973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c3b80995944fca9b42178963cb8dc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3b80995944fca9b42178963cb8dc4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42601 } } }
I20260619 12:09:30.038923 1973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:30.039871 1976 catalog_manager.cc:1347] Generated new cluster ID: 49b44a1e0ed449f5962fa1a21bc00aa4
I20260619 12:09:30.039919 1976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:30.043026 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.031949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.0.201.62:42601)
0619 12:09:30.032138 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.032150 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.032188 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:09:30.032726 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.032734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.032820 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.032966 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.033636 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.035067 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.035958 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.036389 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.036532 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.039533 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.039537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.039537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.039549 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.041492 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.041867 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.041870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.041871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.041961 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.042187 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.042188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.042188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.042421 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.042613 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.042686 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.042823 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260619 12:09:30.058116 1976 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:30.058802 1976 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:30.062667 1976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: Generated new TSK 0
I20260619 12:09:30.062827 1976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:30.094274 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42601
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:30.175043 1993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:30.175328 1993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:30.175369 1993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:30.176509 1993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:30.176580 1993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:30.176618 1993 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:30.176678 1993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:30.178184 1993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:30.178462 1993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:30.178694 1993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:30.183853 1998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:30.183851 1999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:30.183895 2001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:30.184163 1993 server_base.cc:1034] running on GCE node
I20260619 12:09:30.184393 1993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:30.184692 1993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:30.185864 1993 hybrid_clock.cc:648] HybridClock initialized: now 1781870970185851 us; error 27 us; skew 500 ppm
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:30.188534 1993 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:30.189014 1993 webserver.cc:460] Webserver started at http://127.0.201.1:40921/ using document root <none> and password file <none>
I20260619 12:09:30.189265 1993 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:30.189335 1993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:30.189492 1993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:30.190512 1993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "30125d8719574ef7bf99869d0a77f2b4"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "bf393245453c86e95aed0c689a628329"
server_key_iv: "8cc767877b3240f6ed251e5eaa61d04b"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.190884 1993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "30125d8719574ef7bf99869d0a77f2b4"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "bf393245453c86e95aed0c689a628329"
server_key_iv: "8cc767877b3240f6ed251e5eaa61d04b"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.192304 1993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:30.193087 2008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.193287 1993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:30.193405 1993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "30125d8719574ef7bf99869d0a77f2b4"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "bf393245453c86e95aed0c689a628329"
server_key_iv: "8cc767877b3240f6ed251e5eaa61d04b"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.193545 1993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:30.229625 1993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:30.232295 1993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:30.232512 1993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:30.233151 1993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:30.233227 1993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.233261 1993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:30.233285 1993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:30.240686 1993 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:39129
I20260619 12:09:30.240772 2123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:39129 every 8 connection(s)
I20260619 12:09:30.241106 1993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:30.243558 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.233517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:42263 (local address 127.0.201.62:42601)
0619 12:09:30.233583 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.233585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.233781 (+ 196us) server_negotiation.cc:407] Connection header received
0619 12:09:30.234324 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.234325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.234337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.234384 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.235034 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.235648 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.236661 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.236912 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.237007 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.240536 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.240538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.240538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.240546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.242129 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.242544 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.242545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.242545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.242601 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.242863 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.242864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.242865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.242983 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.243065 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.243378 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.243459 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:09:30.243724 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.233503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.201.1:42263)
0619 12:09:30.233653 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.233696 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.234207 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.234454 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.234458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.234692 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:30.234927 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.234932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.235756 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.235757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.236540 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.236542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.237015 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.237016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.237166 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.237798 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:30.237807 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:30.240452 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:30.242232 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.242239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.242241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.242450 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.242679 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.242680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.242681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.242792 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.243057 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:30.243063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:30.243234 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.243360 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.243509 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260619 12:09:30.244196 2124 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42601
I20260619 12:09:30.244275 2124 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:30.244458 2124 heartbeater.cc:511] Master 127.0.201.62:42601 requested a full tablet report, sending...
I20260619 12:09:30.244975 1920 ts_manager.cc:194] Registered new tserver with Master: 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129)
I20260619 12:09:30.245708 1920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:42263
I20260619 12:09:30.249431 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 1993
I20260619 12:09:30.249534 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:30.249739 804 external_mini_cluster.cc:1428] Setting key 9513186f6f16acc370c72642b048a903
I20260619 12:09:30.251020 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.245194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:36665 (local address 127.0.201.62:42601)
0619 12:09:30.245244 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.245245 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.245438 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:09:30.245515 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.245516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.245526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.245581 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.245873 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.246350 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.247128 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.247388 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.247479 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.248238 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.248239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.248239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.248244 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.249793 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.250201 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.250202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.250202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.250259 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.250520 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.250521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.250521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.250628 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.250705 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.250860 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.250935 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:30.251029 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.245237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.201.1:36665)
0619 12:09:30.245372 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.245382 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.245453 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.245585 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.245586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.245749 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:30.245793 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.245794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.246440 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.246441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.247069 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.247071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.247497 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.247498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.247542 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.247895 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:30.247899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:30.248160 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:30.249888 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.249889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.249889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.250115 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.250384 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.250385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.250385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.250448 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.250697 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:30.250698 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:30.250739 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.250835 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.250936 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:30.274111 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42601
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:30.355937 2133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:30.356138 2133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:30.356163 2133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:30.357223 2133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:30.357255 2133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:30.357275 2133 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:30.357291 2133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:30.358767 2133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:30.358942 2133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:30.359114 2133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:30.364099 2138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:30.364130 2141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:30.364131 2139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:30.364360 2133 server_base.cc:1034] running on GCE node
I20260619 12:09:30.364624 2133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:30.364820 2133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:30.365953 2133 hybrid_clock.cc:648] HybridClock initialized: now 1781870970365941 us; error 29 us; skew 500 ppm
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:30.368418 2133 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:30.368870 2133 webserver.cc:460] Webserver started at http://127.0.201.2:45329/ using document root <none> and password file <none>
I20260619 12:09:30.369047 2133 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:30.369094 2133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:30.369182 2133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:30.370136 2133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "a3994ee7fa504ac08d216ad9fa91f7f8"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "d0664b3cbcce7cbc58d3c2dc952fdef7"
server_key_iv: "709a1f674f60c6e67cb710b96f59004f"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.370422 2133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "a3994ee7fa504ac08d216ad9fa91f7f8"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "d0664b3cbcce7cbc58d3c2dc952fdef7"
server_key_iv: "709a1f674f60c6e67cb710b96f59004f"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.371804 2133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:09:30.372496 2148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.372725 2133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:30.372807 2133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "a3994ee7fa504ac08d216ad9fa91f7f8"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "d0664b3cbcce7cbc58d3c2dc952fdef7"
server_key_iv: "709a1f674f60c6e67cb710b96f59004f"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.372861 2133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:30.384061 2133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:30.386003 2133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:30.386111 2133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:30.386615 2133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:30.386654 2133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.386675 2133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:30.386688 2133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:30.393983 2133 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:36755
I20260619 12:09:30.394321 2133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:30.394429 2263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:36755 every 8 connection(s)
I20260619 12:09:30.396598 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.387124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:58195 (local address 127.0.201.62:42601)
0619 12:09:30.387211 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.387213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.387389 (+ 176us) server_negotiation.cc:407] Connection header received
0619 12:09:30.387943 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.387944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.387960 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.388024 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.388757 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.389265 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.390446 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.390655 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.390757 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.393464 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.393466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.393466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.393475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.395242 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.395672 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.395674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.395674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.395727 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.396000 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.396000 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.396000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.396116 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.396166 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.396444 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.396535 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:30.396886 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.387074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.201.2:58195)
0619 12:09:30.387243 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.387284 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.387806 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.388127 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.388132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.388405 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:30.388640 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.388645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.389398 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.389399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.390282 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.390284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.390766 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.390767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.390899 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.391531 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:30.391542 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:30.393338 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:30.395335 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.395340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.395342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.395599 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.395817 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.395818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.395818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.395933 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.396181 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:30.396185 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:30.396300 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.396455 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.396638 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260619 12:09:30.397385 2264 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42601
I20260619 12:09:30.397470 2264 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:30.397730 2264 heartbeater.cc:511] Master 127.0.201.62:42601 requested a full tablet report, sending...
I20260619 12:09:30.398108 1921 ts_manager.cc:194] Registered new tserver with Master: a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2:36755)
I20260619 12:09:30.398520 1921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:58195
I20260619 12:09:30.398699 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 2133
I20260619 12:09:30.398777 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:30.398985 804 external_mini_cluster.cc:1428] Setting key fa4c611696e4569672f9e8f6bf05f4dd
I20260619 12:09:30.407064 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.398494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.201.2:53625)
0619 12:09:30.399269 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.399281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.399367 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.399510 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.399511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.401303 (+ 1792us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:30.401364 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.401365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.402260 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.402261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.402835 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.402837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.403339 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.403340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.403391 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.403917 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:30.403922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:30.404227 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:30.405910 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.405911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.405911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.406080 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.406336 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.406336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.406337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.406400 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.406757 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:30.406757 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:30.406803 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.406887 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.406989 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":89,"threads_started":1}
I20260619 12:09:30.407104 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.399170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:53625 (local address 127.0.201.62:42601)
0619 12:09:30.399257 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.399259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.399340 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:09:30.399437 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.399437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.399447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.399503 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.401446 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.402166 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.402924 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.403268 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.403341 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.404322 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.404323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.404324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.404331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.405824 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.406161 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.406163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.406163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.406216 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.406479 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.406480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.406481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.406684 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.406766 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.406907 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.406984 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:30.425855 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42601
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:30.508208 2273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:30.508440 2273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:30.508479 2273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:30.509620 2273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:30.509693 2273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:30.509727 2273 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:30.509766 2273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:30.511227 2273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:30.511505 2273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:30.511734 2273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:30.516786 2281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:30.516909 2273 server_base.cc:1034] running on GCE node
W20260619 12:09:30.516777 2279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:30.516793 2278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:30.517266 2273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:30.517459 2273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:30.518600 2273 hybrid_clock.cc:648] HybridClock initialized: now 1781870970518581 us; error 36 us; skew 500 ppm
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:30.521103 2273 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:30.521589 2273 webserver.cc:460] Webserver started at http://127.0.201.3:36259/ using document root <none> and password file <none>
I20260619 12:09:30.521829 2273 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:30.521900 2273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:30.522063 2273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:30.523058 2273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "b9c0937186714157b34744155d4789b8"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "ac46bc2fc2e9d088660deb9044e082be"
server_key_iv: "1670a5ed8a2e548c63bd26d0ecfd9c3c"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.523458 2273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "b9c0937186714157b34744155d4789b8"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "ac46bc2fc2e9d088660deb9044e082be"
server_key_iv: "1670a5ed8a2e548c63bd26d0ecfd9c3c"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.524711 2273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:30.525362 2288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.525532 2273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:30.525645 2273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "b9c0937186714157b34744155d4789b8"
format_stamp: "Formatted at 2026-06-19 12:09:30 on dist-test-slave-c910"
server_key: "ac46bc2fc2e9d088660deb9044e082be"
server_key_iv: "1670a5ed8a2e548c63bd26d0ecfd9c3c"
server_key_version: "encryptionkey@0"
I20260619 12:09:30.525727 2273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:30.550303 2273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:30.552795 2273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:30.552973 2273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:30.553543 2273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:30.553606 2273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:30.553649 2273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:30.553679 2273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:30.561251 2273 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:39507
I20260619 12:09:30.561332 2403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:39507 every 8 connection(s)
I20260619 12:09:30.561635 2273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:30.563050 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.553986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:35541 (local address 127.0.201.62:42601)
0619 12:09:30.554062 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.554065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.554247 (+ 182us) server_negotiation.cc:407] Connection header received
0619 12:09:30.554700 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.554702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.554714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.554798 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.555493 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.555987 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.557000 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.557212 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.557302 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.559918 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.559919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.559920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.559928 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.561634 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.562091 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.562092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.562092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.562150 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.562371 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.562371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.562372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.562475 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.562559 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.562857 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.562934 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:09:30.563242 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.553958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.201.3:35541)
0619 12:09:30.554110 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.554150 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.554608 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.554848 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.554851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.555164 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:30.555398 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.555404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.556106 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.556107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.556906 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.556908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.557309 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.557310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.557466 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.557929 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:30.557937 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:30.559805 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:30.561728 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.561733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.561735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.562005 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.562211 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.562212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.562212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.562306 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.562552 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:30.562557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:30.562712 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.562835 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.562990 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":77,"threads_started":1}
I20260619 12:09:30.563725 2404 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42601
I20260619 12:09:30.563798 2404 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:30.563999 2404 heartbeater.cc:511] Master 127.0.201.62:42601 requested a full tablet report, sending...
I20260619 12:09:30.564374 1921 ts_manager.cc:194] Registered new tserver with Master: b9c0937186714157b34744155d4789b8 (127.0.201.3:39507)
I20260619 12:09:30.564874 1921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:35541
I20260619 12:09:30.569176 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.564674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.201.3:41199)
0619 12:09:30.564813 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.564822 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.564889 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.565013 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.565014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.565177 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:30.565253 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.565254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.565853 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.565854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.566335 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.566336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.566635 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.566636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.566680 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.566993 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:30.566997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:30.567326 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:30.568485 (+ 1159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.568485 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.568485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.568636 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.568766 (+ 130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:30.568766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:30.568767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:30.568805 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:30.568955 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:30.568955 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:30.568986 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.569031 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.569113 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260619 12:09:30.569204 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.564719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:41199 (local address 127.0.201.62:42601)
0619 12:09:30.564757 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.564759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.564876 (+ 117us) server_negotiation.cc:407] Connection header received
0619 12:09:30.564945 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.564946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.564955 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.565005 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.565331 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.565770 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.566401 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.566565 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.566621 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.567393 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.567394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.567394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.567400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.568416 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.568693 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.568693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.568694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.568731 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.568845 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.568845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.568845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.568921 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.568961 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.569045 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.569091 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:30.570840 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 2273
I20260619 12:09:30.570940 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:30.571082 804 external_mini_cluster.cc:1428] Setting key 866c9605e8c3faa24c27c1ba6ecaa894
I20260619 12:09:30.571995 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:09:30 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:30.593869 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.587094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.0.201.62:42601)
0619 12:09:30.587161 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.587163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.587172 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:30.587217 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.587218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.587228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.587295 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:30.587841 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.588314 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.589031 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.589206 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.589253 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.591574 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:30.591575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:30.591576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:30.591587 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:30.592862 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.593201 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.593202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.593202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.593249 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:30.593416 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:30.593416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:30.593416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:30.593517 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:30.593596 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.593701 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.593759 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:09:30.595736 1921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:30.597478 1921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:30.603950 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.599799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36755 (local address 127.0.0.1:41892)
0619 12:09:30.600171 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.600183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.600211 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.600506 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.600509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.600512 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:30.600606 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.600607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.601888 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.601889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.602809 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.602811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.603652 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.603736 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.603750 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.603819 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.603870 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":48,"thread_start_us":83,"threads_started":1}
I20260619 12:09:30.603968 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.600073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.0.201.2:36755)
0619 12:09:30.600253 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.600258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.600269 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:30.600312 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.600314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.600331 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.600421 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:30.600686 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.601788 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.602912 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.603550 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603651 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.603732 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.603845 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.603881 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
I20260619 12:09:30.604130 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.599789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39507 (local address 127.0.0.1:48016)
0619 12:09:30.599957 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.599999 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.600067 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.600512 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.600513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.600514 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:30.600600 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.600602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.601878 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.601879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.602988 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.602990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.603843 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.603921 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.603926 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.603989 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.604030 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":74,"thread_start_us":69,"threads_started":1}
I20260619 12:09:30.604432 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.599791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:39129 (local address 127.0.0.1:59682)
0619 12:09:30.600033 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.600042 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.600098 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.600550 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.600550 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.600553 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:30.600625 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.600627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.601538 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.601539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.602784 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.602787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.603841 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.604276 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.604280 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.604336 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.604370 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260619 12:09:30.604432 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.599935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.0.201.3:39507)
0619 12:09:30.600337 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.600340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.600349 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:30.600385 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.600387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.600401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.600681 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:30.600737 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.601794 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603067 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.603766 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603869 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.603936 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.604138 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.604339 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":97,"threads_started":1}
I20260619 12:09:30.604502 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.600136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.0.201.1:39129)
0619 12:09:30.600315 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.600319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.600349 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:09:30.600390 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.600393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.600409 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.600483 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:30.600706 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.601449 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.602874 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.603660 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.603732 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.603805 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.604368 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.604412 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260619 12:09:30.604801 2198 tablet_service.cc:1467] Processing CreateTablet for tablet 9596c8a074e44bcba474b47500e2bf15 (DEFAULT_TABLE table=test-table [id=0a88b01dd3e14fe4af1133843b94bfee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:30.605021 2058 tablet_service.cc:1467] Processing CreateTablet for tablet 9596c8a074e44bcba474b47500e2bf15 (DEFAULT_TABLE table=test-table [id=0a88b01dd3e14fe4af1133843b94bfee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:30.605152 2198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9596c8a074e44bcba474b47500e2bf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.605162 2338 tablet_service.cc:1467] Processing CreateTablet for tablet 9596c8a074e44bcba474b47500e2bf15 (DEFAULT_TABLE table=test-table [id=0a88b01dd3e14fe4af1133843b94bfee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:30.605320 2058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9596c8a074e44bcba474b47500e2bf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.605508 2338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9596c8a074e44bcba474b47500e2bf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.608597 2427 tablet_bootstrap.cc:492] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Bootstrap starting.
I20260619 12:09:30.608978 2428 tablet_bootstrap.cc:492] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Bootstrap starting.
I20260619 12:09:30.609297 2427 tablet_bootstrap.cc:654] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.609594 2428 tablet_bootstrap.cc:654] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.609627 2427 log.cc:824] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:30.609913 2428 log.cc:824] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:30.610225 2429 tablet_bootstrap.cc:492] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Bootstrap starting.
I20260619 12:09:30.610424 2427 tablet_bootstrap.cc:492] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: No bootstrap required, opened a new log
I20260619 12:09:30.610486 2427 ts_tablet_manager.cc:1397] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:30.610772 2429 tablet_bootstrap.cc:654] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.611110 2429 log.cc:824] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:30.611646 2428 tablet_bootstrap.cc:492] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: No bootstrap required, opened a new log
I20260619 12:09:30.611734 2428 ts_tablet_manager.cc:1397] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:09:30.612124 2427 raft_consensus.cc:348] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.612247 2427 raft_consensus.cc:374] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.612268 2427 raft_consensus.cc:729] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0937186714157b34744155d4789b8, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.612394 2427 consensus_queue.cc:260] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.612651 2427 ts_tablet_manager.cc:1428] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:30.612674 2404 heartbeater.cc:503] Master 127.0.201.62:42601 was elected leader, sending a full tablet report...
I20260619 12:09:30.613451 2428 raft_consensus.cc:348] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.613672 2428 raft_consensus.cc:374] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.613720 2428 raft_consensus.cc:729] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3994ee7fa504ac08d216ad9fa91f7f8, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.614152 2429 tablet_bootstrap.cc:492] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: No bootstrap required, opened a new log
I20260619 12:09:30.614238 2429 ts_tablet_manager.cc:1397] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:09:30.615283 2428 consensus_queue.cc:260] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.615629 2264 heartbeater.cc:503] Master 127.0.201.62:42601 was elected leader, sending a full tablet report...
I20260619 12:09:30.615854 2428 ts_tablet_manager.cc:1428] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 12:09:30.615856 2429 raft_consensus.cc:348] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.615988 2429 raft_consensus.cc:374] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.616039 2429 raft_consensus.cc:729] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30125d8719574ef7bf99869d0a77f2b4, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.616165 2429 consensus_queue.cc:260] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.616416 2429 ts_tablet_manager.cc:1428] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:30.616448 2124 heartbeater.cc:503] Master 127.0.201.62:42601 was elected leader, sending a full tablet report...
W20260619 12:09:30.645081 2265 tablet.cc:2369] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:09:30.742185 2125 tablet.cc:2369] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:09:30.812316 2405 tablet.cc:2369] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:30.886754 2434 raft_consensus.cc:482] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:30.886875 2434 raft_consensus.cc:504] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.887421 2434 leader_election.cc:288] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129), b9c0937186714157b34744155d4789b8 (127.0.201.3:39507)
I20260619 12:09:30.890198 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.887673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:39795 (local address 127.0.201.1:39129)
0619 12:09:30.887718 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.887720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.887730 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:30.887801 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.887802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.887814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.887897 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:30.888051 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.888517 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.889469 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.889832 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.889929 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.889938 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.890099 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.890130 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:30.890223 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.887608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:39129 (local address 127.0.201.2:39795)
0619 12:09:30.887671 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.887683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.887728 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.887898 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.887899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.887901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:30.887977 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.887979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.888621 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.888622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.889376 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.889379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.889929 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.889930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.890007 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.890010 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.890091 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.890131 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":28}
I20260619 12:09:30.890332 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.887734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:33065 (local address 127.0.201.3:39507)
0619 12:09:30.887778 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.887780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.887824 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:09:30.887927 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.887928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.887939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.888014 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:30.888191 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.888642 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.889613 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.889966 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.890051 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.890060 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.890234 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.890271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:30.890339 2436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.887623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39507 (local address 127.0.201.2:33065)
0619 12:09:30.887766 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:30.887777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:30.887840 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:30.888033 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:30.888034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:30.888036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:30.888114 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.888115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.888731 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.888732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.889505 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:30.889506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.890054 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.890055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:30.890133 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.890137 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:30.890222 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 12:09:30.890260 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260619 12:09:30.890635 2078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9596c8a074e44bcba474b47500e2bf15" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30125d8719574ef7bf99869d0a77f2b4" is_pre_election: true
I20260619 12:09:30.890653 2358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9596c8a074e44bcba474b47500e2bf15" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0937186714157b34744155d4789b8" is_pre_election: true
I20260619 12:09:30.890786 2358 raft_consensus.cc:2454] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 0.
I20260619 12:09:30.890801 2078 raft_consensus.cc:2454] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 0.
I20260619 12:09:30.891018 2149 leader_election.cc:302] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3994ee7fa504ac08d216ad9fa91f7f8, b9c0937186714157b34744155d4789b8; no voters:
I20260619 12:09:30.891170 2434 raft_consensus.cc:2781] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:30.891247 2434 raft_consensus.cc:482] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:30.891284 2434 raft_consensus.cc:3037] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:30.892138 2434 raft_consensus.cc:504] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.892277 2434 leader_election.cc:288] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 election: Requested vote from peers 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129), b9c0937186714157b34744155d4789b8 (127.0.201.3:39507)
I20260619 12:09:30.892426 2358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9596c8a074e44bcba474b47500e2bf15" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0937186714157b34744155d4789b8"
I20260619 12:09:30.892447 2078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9596c8a074e44bcba474b47500e2bf15" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30125d8719574ef7bf99869d0a77f2b4"
I20260619 12:09:30.892517 2358 raft_consensus.cc:3037] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:30.892513 2078 raft_consensus.cc:3037] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:30.893148 2078 raft_consensus.cc:2454] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 1.
I20260619 12:09:30.893263 2358 raft_consensus.cc:2454] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 1.
I20260619 12:09:30.893322 2150 leader_election.cc:302] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30125d8719574ef7bf99869d0a77f2b4, a3994ee7fa504ac08d216ad9fa91f7f8; no voters:
I20260619 12:09:30.893432 2434 raft_consensus.cc:2781] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:30.893596 2434 raft_consensus.cc:686] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 LEADER]: Becoming Leader. State: Replica: a3994ee7fa504ac08d216ad9fa91f7f8, State: Running, Role: LEADER
I20260619 12:09:30.893702 2434 consensus_queue.cc:237] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } }
I20260619 12:09:30.894588 1921 catalog_manager.cc:5495] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } health_report { overall_health: HEALTHY } } }
I20260619 12:09:30.956929 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:30.954488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.0.201.2:36755)
0619 12:09:30.954560 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:30.954562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:30.954572 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:30.954605 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:30.954606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:30.954624 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:30.954683 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:30.954843 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.955343 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.956169 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:30.956343 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:30.956390 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:30.956592 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:30.956679 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:30.956737 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:30.956854 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:09:30.956871 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:09:30.958668 2358 raft_consensus.cc:1261] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Refusing update from remote peer a3994ee7fa504ac08d216ad9fa91f7f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:30.958688 2078 raft_consensus.cc:1261] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Refusing update from remote peer a3994ee7fa504ac08d216ad9fa91f7f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:30.959024 2437 consensus_queue.cc:1035] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:30.959163 2434 consensus_queue.cc:1035] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:30.961647 2440 mvcc.cc:205] Tried to move back new op lower bound from 7298543497044041728 to 7298543496781279232. Current Snapshot: MvccSnapshot[applied={T|T < 7298543497044041728}]
I20260619 12:09:30.965198 1921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:30.965322 1921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:30.966125 1921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: Sending DeleteTablet for 3 replicas of tablet 9596c8a074e44bcba474b47500e2bf15
I20260619 12:09:30.966487 2198 tablet_service.cc:1514] Processing DeleteTablet for tablet 9596c8a074e44bcba474b47500e2bf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:30 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:41892
I20260619 12:09:30.966704 2338 tablet_service.cc:1514] Processing DeleteTablet for tablet 9596c8a074e44bcba474b47500e2bf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:30 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:48016
I20260619 12:09:30.966816 1921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:30.966958 1921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:30.966974 2058 tablet_service.cc:1514] Processing DeleteTablet for tablet 9596c8a074e44bcba474b47500e2bf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:30 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:59682
I20260619 12:09:30.967211 2449 tablet_replica.cc:330] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: stopping tablet replica
I20260619 12:09:30.967276 2451 tablet_replica.cc:330] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: stopping tablet replica
I20260619 12:09:30.967365 2451 raft_consensus.cc:2229] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:30.967466 2451 raft_consensus.cc:2258] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:30.967314 2449 raft_consensus.cc:2229] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:30.967645 2449 raft_consensus.cc:2258] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:30.966914 2450 tablet_replica.cc:330] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: stopping tablet replica
I20260619 12:09:30.968143 2450 raft_consensus.cc:2229] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:30.968274 2450 raft_consensus.cc:2258] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:30.968539 2198 tablet_service.cc:1467] Processing CreateTablet for tablet c4fc3d5bf0e44b18b25d97baf262f188 (DEFAULT_TABLE table=test-table [id=5c8cf43128994fb3972451bcb5b345aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:30.968659 2198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4fc3d5bf0e44b18b25d97baf262f188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.968796 2338 tablet_service.cc:1467] Processing CreateTablet for tablet c4fc3d5bf0e44b18b25d97baf262f188 (DEFAULT_TABLE table=test-table [id=5c8cf43128994fb3972451bcb5b345aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:30.968816 2450 ts_tablet_manager.cc:1905] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:30.968935 2338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4fc3d5bf0e44b18b25d97baf262f188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.969995 2428 tablet_bootstrap.cc:492] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Bootstrap starting.
I20260619 12:09:30.970338 2427 tablet_bootstrap.cc:492] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Bootstrap starting.
I20260619 12:09:30.970430 2428 tablet_bootstrap.cc:654] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.970726 2427 tablet_bootstrap.cc:654] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.970726 2058 tablet_service.cc:1467] Processing CreateTablet for tablet c4fc3d5bf0e44b18b25d97baf262f188 (DEFAULT_TABLE table=test-table [id=5c8cf43128994fb3972451bcb5b345aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:30.970832 2058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4fc3d5bf0e44b18b25d97baf262f188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:30.971218 2450 ts_tablet_manager.cc:1918] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:30.971293 2450 log.cc:1196] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/9596c8a074e44bcba474b47500e2bf15
I20260619 12:09:30.971681 2428 tablet_bootstrap.cc:492] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: No bootstrap required, opened a new log
I20260619 12:09:30.971738 2428 ts_tablet_manager.cc:1397] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:30.971877 2428 raft_consensus.cc:348] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:30.971971 2428 raft_consensus.cc:374] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.971999 2428 raft_consensus.cc:729] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3994ee7fa504ac08d216ad9fa91f7f8, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.972059 2428 consensus_queue.cc:260] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:30.972169 2428 ts_tablet_manager.cc:1428] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:30.972416 2450 ts_tablet_manager.cc:1939] T 9596c8a074e44bcba474b47500e2bf15 P b9c0937186714157b34744155d4789b8: Deleting consensus metadata
I20260619 12:09:30.972788 2429 tablet_bootstrap.cc:492] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Bootstrap starting.
I20260619 12:09:30.972862 1905 catalog_manager.cc:4841] TS b9c0937186714157b34744155d4789b8 (127.0.201.3:39507): tablet 9596c8a074e44bcba474b47500e2bf15 (table test-table [id=0a88b01dd3e14fe4af1133843b94bfee]) successfully deleted
I20260619 12:09:30.973105 2429 tablet_bootstrap.cc:654] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:30.974005 2429 tablet_bootstrap.cc:492] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: No bootstrap required, opened a new log
I20260619 12:09:30.974064 2429 ts_tablet_manager.cc:1397] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:30.974666 2429 raft_consensus.cc:348] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:30.974736 2429 raft_consensus.cc:374] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.974751 2429 raft_consensus.cc:729] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30125d8719574ef7bf99869d0a77f2b4, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.974819 2429 consensus_queue.cc:260] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:30.974959 2429 ts_tablet_manager.cc:1428] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:30.975618 2451 ts_tablet_manager.cc:1905] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:30.976804 2451 ts_tablet_manager.cc:1918] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:30.976871 2451 log.cc:1196] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/9596c8a074e44bcba474b47500e2bf15
I20260619 12:09:30.977358 2451 ts_tablet_manager.cc:1939] T 9596c8a074e44bcba474b47500e2bf15 P 30125d8719574ef7bf99869d0a77f2b4: Deleting consensus metadata
I20260619 12:09:30.977788 1906 catalog_manager.cc:4841] TS 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129): tablet 9596c8a074e44bcba474b47500e2bf15 (table test-table [id=0a88b01dd3e14fe4af1133843b94bfee]) successfully deleted
I20260619 12:09:30.977932 2449 ts_tablet_manager.cc:1905] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:30.979156 2449 ts_tablet_manager.cc:1918] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:30.979260 2449 log.cc:1196] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/9596c8a074e44bcba474b47500e2bf15
I20260619 12:09:30.979425 2427 tablet_bootstrap.cc:492] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: No bootstrap required, opened a new log
I20260619 12:09:30.979487 2427 ts_tablet_manager.cc:1397] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260619 12:09:30.979550 2449 ts_tablet_manager.cc:1939] T 9596c8a074e44bcba474b47500e2bf15 P a3994ee7fa504ac08d216ad9fa91f7f8: Deleting consensus metadata
I20260619 12:09:30.979663 2427 raft_consensus.cc:348] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:30.979751 2427 raft_consensus.cc:374] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:30.979776 2427 raft_consensus.cc:729] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0937186714157b34744155d4789b8, State: Initialized, Role: FOLLOWER
I20260619 12:09:30.979827 2427 consensus_queue.cc:260] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:30.979928 1907 catalog_manager.cc:4841] TS a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2:36755): tablet 9596c8a074e44bcba474b47500e2bf15 (table test-table [id=0a88b01dd3e14fe4af1133843b94bfee]) successfully deleted
I20260619 12:09:30.979959 2427 ts_tablet_manager.cc:1428] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:31.289067 2433 raft_consensus.cc:482] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:31.289171 2433 raft_consensus.cc:504] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.289474 2433 leader_election.cc:288] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129), a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2:36755)
I20260619 12:09:31.293715 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.289772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:43477 (local address 127.0.201.1:39129)
0619 12:09:31.289816 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.289818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.289923 (+ 105us) server_negotiation.cc:407] Connection header received
0619 12:09:31.290025 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.290026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.290036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.290093 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:31.290348 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.291109 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.292714 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.293273 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.293350 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.293359 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.293601 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.293647 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:09:31.293715 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.289768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:53019 (local address 127.0.201.2:36755)
0619 12:09:31.289816 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.289818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.289923 (+ 105us) server_negotiation.cc:407] Connection header received
0619 12:09:31.289962 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.289963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.289976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.290062 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:31.290348 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.291106 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.292714 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.293273 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.293350 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.293359 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.293601 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.293646 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:31.293702 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.289699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36755 (local address 127.0.201.3:53019)
0619 12:09:31.289827 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:31.289840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:31.289909 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:31.290108 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:31.290109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:31.290112 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:31.290208 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.290210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.291238 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.291239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.292612 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.292615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.293375 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.293376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.293499 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.293502 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:31.293554 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:09:31.293582 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":146,"thread_start_us":63,"threads_started":1}
I20260619 12:09:31.293707 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.289699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:39129 (local address 127.0.201.3:43477)
0619 12:09:31.289827 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:31.289840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:31.289909 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:31.290112 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:31.290112 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:31.290114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:31.290208 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.290210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.291238 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.291239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.292612 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.292615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.293376 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.293377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.293462 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.293466 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:31.293532 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 12:09:31.293582 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":71,"thread_start_us":60,"threads_started":1}
I20260619 12:09:31.293970 2078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fc3d5bf0e44b18b25d97baf262f188" candidate_uuid: "b9c0937186714157b34744155d4789b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30125d8719574ef7bf99869d0a77f2b4" is_pre_election: true
I20260619 12:09:31.294054 2078 raft_consensus.cc:2454] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9c0937186714157b34744155d4789b8 in term 0.
I20260619 12:09:31.294179 2218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fc3d5bf0e44b18b25d97baf262f188" candidate_uuid: "b9c0937186714157b34744155d4789b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" is_pre_election: true
I20260619 12:09:31.294241 2290 leader_election.cc:302] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30125d8719574ef7bf99869d0a77f2b4, b9c0937186714157b34744155d4789b8; no voters:
I20260619 12:09:31.294305 2218 raft_consensus.cc:2454] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9c0937186714157b34744155d4789b8 in term 0.
I20260619 12:09:31.294442 2433 raft_consensus.cc:2781] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:31.294494 2433 raft_consensus.cc:482] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:31.294514 2433 raft_consensus.cc:3037] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:31.295279 2433 raft_consensus.cc:504] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.295449 2433 leader_election.cc:288] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [CANDIDATE]: Term 1 election: Requested vote from peers 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129), a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2:36755)
I20260619 12:09:31.295640 2078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fc3d5bf0e44b18b25d97baf262f188" candidate_uuid: "b9c0937186714157b34744155d4789b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30125d8719574ef7bf99869d0a77f2b4"
I20260619 12:09:31.295707 2078 raft_consensus.cc:3037] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:31.295642 2218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fc3d5bf0e44b18b25d97baf262f188" candidate_uuid: "b9c0937186714157b34744155d4789b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8"
I20260619 12:09:31.295743 2218 raft_consensus.cc:3037] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:31.296376 2078 raft_consensus.cc:2454] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9c0937186714157b34744155d4789b8 in term 1.
I20260619 12:09:31.296533 2218 raft_consensus.cc:2454] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9c0937186714157b34744155d4789b8 in term 1.
I20260619 12:09:31.296561 2290 leader_election.cc:302] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30125d8719574ef7bf99869d0a77f2b4, b9c0937186714157b34744155d4789b8; no voters:
I20260619 12:09:31.296639 2433 raft_consensus.cc:2781] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:31.296765 2433 raft_consensus.cc:686] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 1 LEADER]: Becoming Leader. State: Replica: b9c0937186714157b34744155d4789b8, State: Running, Role: LEADER
I20260619 12:09:31.296847 2433 consensus_queue.cc:237] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.297578 1920 catalog_manager.cc:5495] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9c0937186714157b34744155d4789b8 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "b9c0937186714157b34744155d4789b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } health_report { overall_health: HEALTHY } } }
Jun 19 12:09:31 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:31.354642 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.347873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42601 (local address 127.0.0.1:40342)
0619 12:09:31.348104 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:31.348115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:31.348218 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:31.348392 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:31.348393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:31.348730 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:31.348773 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.348775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.349450 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.349451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.349981 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.349982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.350368 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.350369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.350421 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.350923 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:31.350927 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:31.352258 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:31.353726 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:31.353731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:31.353733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:31.353924 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:31.354110 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:31.354111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:31.354111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:31.354165 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:31.354372 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:31.354377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:31.354416 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 12:09:31.354466 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:09:31.354549 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":124,"threads_started":1}
I20260619 12:09:31.354693 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.347955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.0.201.62:42601)
0619 12:09:31.348105 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.348107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.348196 (+ 89us) server_negotiation.cc:407] Connection header received
0619 12:09:31.348290 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.348291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.348307 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.348388 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:31.348862 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.349356 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.350071 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.350258 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.350369 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.352346 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:31.352347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:31.352347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:31.352353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:31.353655 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:31.353994 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:31.353995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:31.353995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:31.354048 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:31.354221 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:31.354221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:31.354222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:31.354318 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:31.354366 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.354513 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.354597 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260619 12:09:31.356736 1920 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:40342:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:09:31.356945 1920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:31.358345 2058 tablet_service.cc:1467] Processing CreateTablet for tablet b2c9b47b7cfd4865847311f38f64af51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6502d4dc9bd34be09e8e09e4be9dc02b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:31.358420 2338 tablet_service.cc:1467] Processing CreateTablet for tablet b2c9b47b7cfd4865847311f38f64af51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6502d4dc9bd34be09e8e09e4be9dc02b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:31.358408 2198 tablet_service.cc:1467] Processing CreateTablet for tablet b2c9b47b7cfd4865847311f38f64af51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6502d4dc9bd34be09e8e09e4be9dc02b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:31.358511 2058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c9b47b7cfd4865847311f38f64af51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:31.358565 2338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c9b47b7cfd4865847311f38f64af51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:31.358582 2198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c9b47b7cfd4865847311f38f64af51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:31.359897 2429 tablet_bootstrap.cc:492] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: Bootstrap starting.
I20260619 12:09:31.360143 2428 tablet_bootstrap.cc:492] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: Bootstrap starting.
I20260619 12:09:31.360316 2429 tablet_bootstrap.cc:654] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:31.360455 2428 tablet_bootstrap.cc:654] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:31.360710 2427 tablet_bootstrap.cc:492] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: Bootstrap starting.
I20260619 12:09:31.361004 2429 tablet_bootstrap.cc:492] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: No bootstrap required, opened a new log
I20260619 12:09:31.361079 2429 ts_tablet_manager.cc:1397] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:31.361481 2429 raft_consensus.cc:348] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.361560 2429 raft_consensus.cc:374] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:31.361580 2429 raft_consensus.cc:729] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30125d8719574ef7bf99869d0a77f2b4, State: Initialized, Role: FOLLOWER
I20260619 12:09:31.361555 2427 tablet_bootstrap.cc:654] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:31.361647 2429 consensus_queue.cc:260] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.361749 2429 ts_tablet_manager.cc:1428] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:31.362030 2435 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.362133 2435 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.362264 2427 tablet_bootstrap.cc:492] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: No bootstrap required, opened a new log
I20260619 12:09:31.362318 2427 ts_tablet_manager.cc:1397] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:31.362454 2428 tablet_bootstrap.cc:492] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: No bootstrap required, opened a new log
I20260619 12:09:31.362497 2428 ts_tablet_manager.cc:1397] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:31.362452 2427 raft_consensus.cc:348] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.362531 2427 raft_consensus.cc:374] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:31.362562 2427 raft_consensus.cc:729] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c0937186714157b34744155d4789b8, State: Initialized, Role: FOLLOWER
I20260619 12:09:31.362609 2427 consensus_queue.cc:260] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.362615 2428 raft_consensus.cc:348] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.362674 2428 raft_consensus.cc:374] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:31.362699 2427 ts_tablet_manager.cc:1428] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:31.362705 2428 raft_consensus.cc:729] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3994ee7fa504ac08d216ad9fa91f7f8, State: Initialized, Role: FOLLOWER
I20260619 12:09:31.362746 2428 consensus_queue.cc:260] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.362754 2433 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.362833 2428 ts_tablet_manager.cc:1428] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:31.362838 2433 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.363030 2437 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.363114 2437 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.689989 2437 raft_consensus.cc:482] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:31.690121 2437 raft_consensus.cc:504] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.690330 2437 leader_election.cc:288] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129), b9c0937186714157b34744155d4789b8 (127.0.201.3:39507)
I20260619 12:09:31.690562 2358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c9b47b7cfd4865847311f38f64af51" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0937186714157b34744155d4789b8" is_pre_election: true
I20260619 12:09:31.690651 2358 raft_consensus.cc:2454] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 0.
I20260619 12:09:31.690593 2078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c9b47b7cfd4865847311f38f64af51" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30125d8719574ef7bf99869d0a77f2b4" is_pre_election: true
I20260619 12:09:31.690713 2078 raft_consensus.cc:2454] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 0.
I20260619 12:09:31.690837 2149 leader_election.cc:302] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3994ee7fa504ac08d216ad9fa91f7f8, b9c0937186714157b34744155d4789b8; no voters:
I20260619 12:09:31.690975 2437 raft_consensus.cc:2781] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:31.691030 2437 raft_consensus.cc:482] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:31.691057 2437 raft_consensus.cc:3037] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:31.691843 2437 raft_consensus.cc:504] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.691996 2437 leader_election.cc:288] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 election: Requested vote from peers 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129), b9c0937186714157b34744155d4789b8 (127.0.201.3:39507)
I20260619 12:09:31.692160 2078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c9b47b7cfd4865847311f38f64af51" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30125d8719574ef7bf99869d0a77f2b4"
I20260619 12:09:31.692154 2358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c9b47b7cfd4865847311f38f64af51" candidate_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c0937186714157b34744155d4789b8"
I20260619 12:09:31.692225 2358 raft_consensus.cc:3037] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:31.692247 2078 raft_consensus.cc:3037] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:31.692799 2078 raft_consensus.cc:2454] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 1.
I20260619 12:09:31.692858 2358 raft_consensus.cc:2454] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3994ee7fa504ac08d216ad9fa91f7f8 in term 1.
I20260619 12:09:31.693009 2150 leader_election.cc:302] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30125d8719574ef7bf99869d0a77f2b4, a3994ee7fa504ac08d216ad9fa91f7f8; no voters:
I20260619 12:09:31.693120 2437 raft_consensus.cc:2781] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:31.693233 2437 raft_consensus.cc:686] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 LEADER]: Becoming Leader. State: Replica: a3994ee7fa504ac08d216ad9fa91f7f8, State: Running, Role: LEADER
I20260619 12:09:31.693306 2437 consensus_queue.cc:237] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } }
I20260619 12:09:31.693570 2469 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: TxnStatusTablet state changed. Reason: New leader a3994ee7fa504ac08d216ad9fa91f7f8. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.693662 2469 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:31.693820 2471 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:09:31.693928 2471 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:09:31.693882 1921 catalog_manager.cc:5495] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:31.715118 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.712372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36755 (local address 127.0.0.1:53564)
0619 12:09:31.712463 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:31.712476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:31.712534 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:31.712693 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:31.712694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:31.712696 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:31.712740 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.712742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.713631 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.713632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.714202 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.714203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.714629 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.714630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.714687 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.714708 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.714924 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.714933 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:09:31.714980 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:09:31.715031 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 12:09:31.715101 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.712467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.0.201.2:36755)
0619 12:09:31.712536 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.712538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.712549 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:31.712612 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.712613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.712625 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.712678 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:31.712803 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.713550 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.714315 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.714510 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.714630 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.714801 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.714845 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.714921 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.714986 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.715006 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:31.716197 2358 raft_consensus.cc:1261] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Refusing update from remote peer a3994ee7fa504ac08d216ad9fa91f7f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:31.716204 2078 raft_consensus.cc:1261] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Refusing update from remote peer a3994ee7fa504ac08d216ad9fa91f7f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:31.716496 2437 consensus_queue.cc:1035] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:31.716614 2469 consensus_queue.cc:1035] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:31.717001 2433 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: TxnStatusTablet state changed. Reason: New leader a3994ee7fa504ac08d216ad9fa91f7f8. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.717108 2433 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.717338 2435 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: TxnStatusTablet state changed. Reason: New leader a3994ee7fa504ac08d216ad9fa91f7f8. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.717417 2435 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.718271 2437 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.718348 2437 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:31.718428 2437 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.718477 2437 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P a3994ee7fa504ac08d216ad9fa91f7f8: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:31.718873 2435 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.718969 2435 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P 30125d8719574ef7bf99869d0a77f2b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.719301 2477 mvcc.cc:205] Tried to move back new op lower bound from 7298543500147834880 to 7298543500056162304. Current Snapshot: MvccSnapshot[applied={T|T < 7298543500147834880}]
I20260619 12:09:31.719460 2433 tablet_replica.cc:439] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 } } peers { permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 } } peers { permanent_uuid: "b9c0937186714157b34744155d4789b8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39507 } } }
I20260619 12:09:31.719534 2433 tablet_replica.cc:442] T b2c9b47b7cfd4865847311f38f64af51 P b9c0937186714157b34744155d4789b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:31.719956 2475 mvcc.cc:205] Tried to move back new op lower bound from 7298543500147834880 to 7298543500056162304. Current Snapshot: MvccSnapshot[applied={T|T < 7298543500147834880}]
I20260619 12:09:31.722599 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.719891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.0.201.3:39507)
0619 12:09:31.720313 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.720315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.720321 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:09:31.720352 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.720352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.720368 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.720467 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:31.720579 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.721040 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.721801 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.722014 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.722098 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.722267 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.722371 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.722461 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.722500 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.722521 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":46,"threads_started":1}
I20260619 12:09:31.727764 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.724969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:41691 (local address 127.0.201.2:36755)
0619 12:09:31.725008 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.725010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.725175 (+ 165us) server_negotiation.cc:407] Connection header received
0619 12:09:31.725223 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.725224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.725232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.725276 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:31.725369 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.725928 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.727023 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.727299 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.727367 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.727551 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.727574 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.727611 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.727686 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.727710 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:31.727788 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.724934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36755 (local address 127.0.201.3:41691)
0619 12:09:31.725100 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:31.725111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:31.725170 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:31.725279 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:31.725280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:31.725281 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:31.725318 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.725319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.726000 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.726001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.726949 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.726951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.727396 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.727397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.727457 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.727472 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.727612 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.727613 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:31.727660 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:09:31.727702 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260619 12:09:31.730012 2078 raft_consensus.cc:1261] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Refusing update from remote peer b9c0937186714157b34744155d4789b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:31.730032 2218 raft_consensus.cc:1261] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Refusing update from remote peer b9c0937186714157b34744155d4789b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:31.730247 2457 consensus_queue.cc:1035] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30125d8719574ef7bf99869d0a77f2b4" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:31.730360 2433 consensus_queue.cc:1035] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3994ee7fa504ac08d216ad9fa91f7f8" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:31.738543 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.735987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:60607 (local address 127.0.201.3:39507)
0619 12:09:31.736115 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.736117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.736175 (+ 58us) server_negotiation.cc:407] Connection header received
0619 12:09:31.736244 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.736245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.736254 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.736304 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:31.736412 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.736852 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.737764 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.738054 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.738142 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.738329 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.738361 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.738404 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.738459 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.738478 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:09:31.738574 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.735949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39507 (local address 127.0.201.2:60607)
0619 12:09:31.736127 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:31.736138 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:31.736190 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:31.736307 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:31.736307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:31.736308 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:31.736348 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.736349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.736937 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.736938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.737704 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:31.737707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.738143 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.738144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:31.738198 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.738211 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:31.738416 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:31.738417 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:31.738459 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 12:09:31.738500 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260619 12:09:31.749213 1921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:31.749306 1921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:31.749890 1921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91c3b80995944fca9b42178963cb8dc4: Sending DeleteTablet for 3 replicas of tablet c4fc3d5bf0e44b18b25d97baf262f188
I20260619 12:09:31.750131 2198 tablet_service.cc:1514] Processing DeleteTablet for tablet c4fc3d5bf0e44b18b25d97baf262f188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:31 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:41892
I20260619 12:09:31.750339 2338 tablet_service.cc:1514] Processing DeleteTablet for tablet c4fc3d5bf0e44b18b25d97baf262f188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:31 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:48016
I20260619 12:09:31.750492 2058 tablet_service.cc:1514] Processing DeleteTablet for tablet c4fc3d5bf0e44b18b25d97baf262f188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:31 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:59682
I20260619 12:09:31.751044 2498 tablet_replica.cc:330] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: stopping tablet replica
I20260619 12:09:31.751125 2496 tablet_replica.cc:330] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: stopping tablet replica
I20260619 12:09:31.751132 2498 raft_consensus.cc:2229] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:31.751212 2496 raft_consensus.cc:2229] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:31.751268 2496 raft_consensus.cc:2258] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:31.751372 2497 tablet_replica.cc:330] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: stopping tablet replica
I20260619 12:09:31.751369 2498 raft_consensus.cc:2258] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:31.751426 2497 raft_consensus.cc:2229] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:31.751467 2497 raft_consensus.cc:2258] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:31.751741 2498 ts_tablet_manager.cc:1905] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:31.752051 2497 ts_tablet_manager.cc:1905] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:31.753650 2498 ts_tablet_manager.cc:1918] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:31.753749 2496 ts_tablet_manager.cc:1905] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:31.753767 2498 log.cc:1196] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/c4fc3d5bf0e44b18b25d97baf262f188
I20260619 12:09:31.754086 2498 ts_tablet_manager.cc:1939] T c4fc3d5bf0e44b18b25d97baf262f188 P b9c0937186714157b34744155d4789b8: Deleting consensus metadata
I20260619 12:09:31.754412 1905 catalog_manager.cc:4841] TS b9c0937186714157b34744155d4789b8 (127.0.201.3:39507): tablet c4fc3d5bf0e44b18b25d97baf262f188 (table test-table [id=5c8cf43128994fb3972451bcb5b345aa]) successfully deleted
I20260619 12:09:31.755309 2496 ts_tablet_manager.cc:1918] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:31.755369 2496 log.cc:1196] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/c4fc3d5bf0e44b18b25d97baf262f188
I20260619 12:09:31.755326 2497 ts_tablet_manager.cc:1918] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:31.755514 2497 log.cc:1196] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/c4fc3d5bf0e44b18b25d97baf262f188
I20260619 12:09:31.755579 2496 ts_tablet_manager.cc:1939] T c4fc3d5bf0e44b18b25d97baf262f188 P a3994ee7fa504ac08d216ad9fa91f7f8: Deleting consensus metadata
I20260619 12:09:31.755971 1907 catalog_manager.cc:4841] TS a3994ee7fa504ac08d216ad9fa91f7f8 (127.0.201.2:36755): tablet c4fc3d5bf0e44b18b25d97baf262f188 (table test-table [id=5c8cf43128994fb3972451bcb5b345aa]) successfully deleted
I20260619 12:09:31.756232 2497 ts_tablet_manager.cc:1939] T c4fc3d5bf0e44b18b25d97baf262f188 P 30125d8719574ef7bf99869d0a77f2b4: Deleting consensus metadata
I20260619 12:09:31.756534 1906 catalog_manager.cc:4841] TS 30125d8719574ef7bf99869d0a77f2b4 (127.0.201.1:39129): tablet c4fc3d5bf0e44b18b25d97baf262f188 (table test-table [id=5c8cf43128994fb3972451bcb5b345aa]) successfully deleted
Jun 19 12:09:31 dist-test-slave-c910 krb5kdc[1876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870970, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:31.766324 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.751699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54320 (local address 127.0.201.1:39129)
0619 12:09:31.751733 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.751735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.756569 (+ 4834us) server_negotiation.cc:407] Connection header received
0619 12:09:31.756674 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.756675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.756687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.756747 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:31.757397 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.758132 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.759178 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.759496 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.759565 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.763000 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:31.763009 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:31.763009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:31.763016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:31.765025 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:31.765466 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:31.765470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:31.765470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:31.765551 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:31.765764 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:31.765764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:31.765765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:31.765934 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:31.766075 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.766157 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.766233 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 12:09:31.766793 2118 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54320
I20260619 12:09:31.775274 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:31.767743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.0.201.62:42601)
0619 12:09:31.767808 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:31.767810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:31.767934 (+ 124us) server_negotiation.cc:407] Connection header received
0619 12:09:31.767994 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:31.767994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:31.768008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:31.768066 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:31.768668 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.769459 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.770439 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:31.770760 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:31.770831 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:31.772284 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:31.772286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:31.772286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:31.772293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:31.773907 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:31.774293 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:31.774294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:31.774294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:31.774352 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:31.774565 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:31.774566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:31.774566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:31.774695 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:31.774794 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:31.775039 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 12:09:31.775144 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 12:09:31.775673 1921 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60368
I20260619 12:09:31.776471 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1993
I20260619 12:09:31.782096 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 2133
I20260619 12:09:31.788033 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 2273
I20260619 12:09:31.792665 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 1889
2026-06-19T12:09:31Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3031 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:31 dist-test-slave-c910 krb5kdc[2505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:31 dist-test-slave-c910 krb5kdc[2505](info): set up 2 sockets
Jun 19 12:09:31 dist-test-slave-c910 krb5kdc[2505](info): commencing operation
krb5kdc: starting...
W20260619 12:09:32.826785 804 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:32 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:32.939415 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44317
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:44317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:33.017795 2518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:33.018041 2518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:33.018079 2518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:33.019258 2518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:33.019320 2518 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:09:33.019346 2518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:33.019367 2518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:33.019414 2518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:33.019469 2518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:33.020948 2518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:44317
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:33.021209 2518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:33.021438 2518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:09:33.026376 2518 server_base.cc:1034] running on GCE node
W20260619 12:09:33.026357 2523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.026352 2524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.026479 2526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:33.026803 2518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:33.027086 2518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:33.028247 2518 hybrid_clock.cc:648] HybridClock initialized: now 1781870973028251 us; error 44 us; skew 500 ppm
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:33.035929 2518 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:33.036441 2518 webserver.cc:460] Webserver started at http://127.0.201.62:34543/ using document root <none> and password file <none>
I20260619 12:09:33.036669 2518 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:33.036736 2518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:33.036891 2518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:33.037981 2518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "322445dd91aa45058adfdf45cd94ad85"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "6d62390f0a77cd07b1bd406adb00663c"
server_key_iv: "64a3d992b2d7befcc488c760e2eb14be"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.038353 2518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "322445dd91aa45058adfdf45cd94ad85"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "6d62390f0a77cd07b1bd406adb00663c"
server_key_iv: "64a3d992b2d7befcc488c760e2eb14be"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.039783 2518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:33.040494 2533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.040664 2518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:33.040768 2518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "322445dd91aa45058adfdf45cd94ad85"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "6d62390f0a77cd07b1bd406adb00663c"
server_key_iv: "64a3d992b2d7befcc488c760e2eb14be"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.040839 2518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:33.056948 2518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:33.059015 2518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:33.059239 2518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:33.064093 2518 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:44317
I20260619 12:09:33.064139 2595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:44317 every 8 connection(s)
I20260619 12:09:33.064464 2518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:33.065089 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.067181 2596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Bootstrap starting.
I20260619 12:09:33.067764 2596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.067993 2596 log.cc:824] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:33.068615 2596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: No bootstrap required, opened a new log
I20260619 12:09:33.070541 2596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "322445dd91aa45058adfdf45cd94ad85" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44317 } }
I20260619 12:09:33.070668 2596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.070690 2596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 322445dd91aa45058adfdf45cd94ad85, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.070784 2596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "322445dd91aa45058adfdf45cd94ad85" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44317 } }
I20260619 12:09:33.070864 2596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:33.070899 2596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:33.070948 2596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:33.071508 2596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "322445dd91aa45058adfdf45cd94ad85" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44317 } }
I20260619 12:09:33.071620 2596 leader_election.cc:302] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 322445dd91aa45058adfdf45cd94ad85; no voters:
I20260619 12:09:33.071782 2596 leader_election.cc:288] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:33.071816 2599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:33.072026 2596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:33.072058 2599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [term 1 LEADER]: Becoming Leader. State: Replica: 322445dd91aa45058adfdf45cd94ad85, State: Running, Role: LEADER
I20260619 12:09:33.072139 2599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "322445dd91aa45058adfdf45cd94ad85" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44317 } }
I20260619 12:09:33.072516 2601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 322445dd91aa45058adfdf45cd94ad85. Latest consensus state: current_term: 1 leader_uuid: "322445dd91aa45058adfdf45cd94ad85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "322445dd91aa45058adfdf45cd94ad85" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44317 } } }
I20260619 12:09:33.072544 2600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "322445dd91aa45058adfdf45cd94ad85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "322445dd91aa45058adfdf45cd94ad85" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44317 } } }
I20260619 12:09:33.072674 2601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:33.072685 2600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:33.073102 2608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:33.073597 2608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:33.073877 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 2518
I20260619 12:09:33.073967 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:33.074160 804 external_mini_cluster.cc:1428] Setting key 47481325205de72d9b976a40f12a4c16
I20260619 12:09:33.075309 2608 catalog_manager.cc:1347] Generated new cluster ID: cbb08198c65e4144b1481db6029c199a
I20260619 12:09:33.075369 2608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:33.085680 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.074724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.0.201.62:44317)
0619 12:09:33.074885 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.074895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.074925 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:09:33.075051 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.075059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.075114 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.075993 (+ 879us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.076083 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.077456 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.078542 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.079130 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.079245 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.082055 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.082059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.082060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.082072 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.084069 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.084432 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.084435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.084435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.084542 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.084736 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.084737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.084737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.085056 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.085208 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.085301 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.085441 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260619 12:09:33.087078 2608 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:33.087626 2608 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:33.091653 2608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Generated new TSK 0
I20260619 12:09:33.091825 2608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:33.123664 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:44317
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:33.204455 2622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:33.204671 2622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:33.204697 2622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:33.205890 2622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:33.205933 2622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:33.205955 2622 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:09:33.205966 2622 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:33.206012 2622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:33.207625 2622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:44317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:33.207878 2622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:33.208129 2622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:33.213038 2627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.212997 2628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.213061 2630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:33.213361 2622 server_base.cc:1034] running on GCE node
I20260619 12:09:33.213562 2622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:33.213841 2622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:33.214998 2622 hybrid_clock.cc:648] HybridClock initialized: now 1781870973214985 us; error 26 us; skew 500 ppm
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:33.217450 2622 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:33.217870 2622 webserver.cc:460] Webserver started at http://127.0.201.1:34751/ using document root <none> and password file <none>
I20260619 12:09:33.218101 2622 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:33.218169 2622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:33.218322 2622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:33.219455 2622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "90e95e347c604116a8c3e31cded4f683"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "0092f348755ee5052b913b38ae02d3aa"
server_key_iv: "e50103db0c799bb405f91fd86204d5f3"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.219933 2622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "90e95e347c604116a8c3e31cded4f683"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "0092f348755ee5052b913b38ae02d3aa"
server_key_iv: "e50103db0c799bb405f91fd86204d5f3"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.221194 2622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:33.221956 2637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.222146 2622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:33.222242 2622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "90e95e347c604116a8c3e31cded4f683"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "0092f348755ee5052b913b38ae02d3aa"
server_key_iv: "e50103db0c799bb405f91fd86204d5f3"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.222325 2622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:33.246435 2622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:33.248487 2622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:33.248665 2622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:33.249198 2622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:33.249264 2622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.249307 2622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:33.249336 2622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:33.256589 2622 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:38315
I20260619 12:09:33.256784 2752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:38315 every 8 connection(s)
I20260619 12:09:33.257015 2622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:33.258751 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 2622
I20260619 12:09:33.258848 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:33.259033 804 external_mini_cluster.cc:1428] Setting key 2ab8d9625f74cf2f01bb11128428f980
I20260619 12:09:33.259176 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.249761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:35599 (local address 127.0.201.62:44317)
0619 12:09:33.249815 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.249818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.250050 (+ 232us) server_negotiation.cc:407] Connection header received
0619 12:09:33.250718 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.250719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.250735 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.250807 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.251831 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.252389 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.253420 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.253659 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.253736 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.256182 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.256184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.256184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.256192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.257838 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.258231 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.258232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.258232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.258281 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.258489 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.258489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.258490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.258608 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.258658 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.258989 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.259058 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:09:33.259420 2646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.249748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.201.1:35599)
0619 12:09:33.249925 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.249959 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.250464 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.250927 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.250932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.251368 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.251687 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.251694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.252509 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.252510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.253321 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.253324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.253766 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.253767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.253903 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.254474 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.254482 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.256092 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.257928 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.257934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.257935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.258164 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.258337 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.258337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.258337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.258425 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.258671 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.258677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.258847 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.258967 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.259145 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260619 12:09:33.260010 2753 heartbeater.cc:347] Connected to a master server at 127.0.201.62:44317
I20260619 12:09:33.260119 2753 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:33.260322 2753 heartbeater.cc:511] Master 127.0.201.62:44317 requested a full tablet report, sending...
I20260619 12:09:33.260799 2550 ts_manager.cc:194] Registered new tserver with Master: 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315)
I20260619 12:09:33.261564 2550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:35599
I20260619 12:09:33.267427 2760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.260995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.201.1:45653)
0619 12:09:33.261806 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.261817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.261871 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.262321 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.262322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.262464 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.262516 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.262517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.263162 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.263163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.263731 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.263733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.264128 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.264128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.264209 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.264677 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.264681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.264960 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.266420 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.266421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.266421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.266586 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.266787 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.266788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.266788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.266849 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.267075 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.267076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.267120 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.267234 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.267330 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
I20260619 12:09:33.267423 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.261265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:45653 (local address 127.0.201.62:44317)
0619 12:09:33.261330 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.261333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.261860 (+ 527us) server_negotiation.cc:407] Connection header received
0619 12:09:33.261923 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.261924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.261935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.261979 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.262578 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.263065 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.263828 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.264032 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.264126 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.265048 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.265050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.265050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.265057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.266332 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.266667 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.266668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.266668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.266718 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.266910 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.266911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.266912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.267013 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.267083 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.267251 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.267330 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:33.284003 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:44317
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:33.367039 2762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:33.367295 2762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:33.367321 2762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:33.368470 2762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:33.368517 2762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:33.368538 2762 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:09:33.368551 2762 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:33.368567 2762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:33.369926 2762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:44317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:33.370108 2762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:33.370277 2762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:33.375169 2768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.375263 2767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.375278 2770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:33.375463 2762 server_base.cc:1034] running on GCE node
I20260619 12:09:33.375674 2762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:33.375898 2762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:33.377053 2762 hybrid_clock.cc:648] HybridClock initialized: now 1781870973377035 us; error 38 us; skew 500 ppm
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:33.379469 2762 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:33.379906 2762 webserver.cc:460] Webserver started at http://127.0.201.2:39285/ using document root <none> and password file <none>
I20260619 12:09:33.380102 2762 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:33.380142 2762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:33.380226 2762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:33.381181 2762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "73aa358785544f6b86db4527961a7adc"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "55dd38cba10d32650bc5d7219abafb05"
server_key_iv: "c6c210d22e019ae8f90448dfba458810"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.381537 2762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "73aa358785544f6b86db4527961a7adc"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "55dd38cba10d32650bc5d7219abafb05"
server_key_iv: "c6c210d22e019ae8f90448dfba458810"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.382745 2762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:09:33.383407 2777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.383565 2762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.383649 2762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "73aa358785544f6b86db4527961a7adc"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "55dd38cba10d32650bc5d7219abafb05"
server_key_iv: "c6c210d22e019ae8f90448dfba458810"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.383700 2762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:33.404448 2762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:33.406243 2762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:33.406352 2762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:33.406809 2762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:33.406847 2762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.406872 2762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:33.406924 2762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.413637 2762 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:42943
I20260619 12:09:33.413705 2892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:42943 every 8 connection(s)
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:33.414096 2762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:33.417066 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.407384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:53479 (local address 127.0.201.62:44317)
0619 12:09:33.407464 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.407467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.407679 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:09:33.408162 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.408164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.408178 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.408261 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.409127 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.409959 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.411534 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.411857 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.411943 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.414527 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.414529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.414530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.414537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.415708 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.416137 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.416138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.416139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.416195 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.416437 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.416438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.416438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.416557 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.416632 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.416893 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.416971 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:09:33.417255 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.407364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.201.2:53479)
0619 12:09:33.407542 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.407581 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.408060 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.408350 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.408354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.408728 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.409015 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.409022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.410076 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.410078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.411439 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.411441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.411950 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.411951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.412084 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.412650 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.412678 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.414442 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.415818 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.415824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.415826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.416051 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.416259 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.416260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.416260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.416370 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.416623 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.416629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.416766 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.416871 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.417033 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260619 12:09:33.417737 2893 heartbeater.cc:347] Connected to a master server at 127.0.201.62:44317
I20260619 12:09:33.417824 2893 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:33.418021 2893 heartbeater.cc:511] Master 127.0.201.62:44317 requested a full tablet report, sending...
I20260619 12:09:33.418401 2549 ts_manager.cc:194] Registered new tserver with Master: 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943)
I20260619 12:09:33.418825 2549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:53479
I20260619 12:09:33.422011 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 2762
I20260619 12:09:33.422119 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:33.422295 804 external_mini_cluster.cc:1428] Setting key 7ff712e18b27184f21effd0bb090d12f
I20260619 12:09:33.424418 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.418876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54969 (local address 127.0.201.62:44317)
0619 12:09:33.418916 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.418918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.419022 (+ 104us) server_negotiation.cc:407] Connection header received
0619 12:09:33.419090 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.419091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.419100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.419145 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.419473 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.419938 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.420676 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.420879 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.420959 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.421734 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.421735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.421735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.421741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.423288 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.423636 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.423638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.423638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.423691 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.423890 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.423891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.423891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.424034 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.424112 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.424258 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.424330 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:33.424434 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.418794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.201.2:54969)
0619 12:09:33.418972 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.418982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.419036 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.419148 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.419149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.419341 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.419390 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.419392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.420032 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.420033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.420595 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.420596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.420961 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.420962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.421034 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.421380 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.421384 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.421640 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.423382 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.423382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.423383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.423553 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.423762 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.423763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.423764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.423820 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.424103 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.424104 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.424149 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.424235 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.424331 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:33.446663 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:44317
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:33.526245 2902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:33.526443 2902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:33.526466 2902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:33.527580 2902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:33.527617 2902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:33.527638 2902 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:09:33.527650 2902 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:33.527666 2902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:33.529019 2902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:44317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:33.529209 2902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:33.529385 2902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:33.534401 2907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:33.534559 2902 server_base.cc:1034] running on GCE node
W20260619 12:09:33.534449 2910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:33.534652 2908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:33.534866 2902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:33.535140 2902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:33.536324 2902 hybrid_clock.cc:648] HybridClock initialized: now 1781870973536305 us; error 27 us; skew 500 ppm
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:33.538846 2902 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:33.539366 2902 webserver.cc:460] Webserver started at http://127.0.201.3:34859/ using document root <none> and password file <none>
I20260619 12:09:33.539619 2902 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:33.539686 2902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:33.539845 2902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:33.540891 2902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "b8d5e4c73750483ca02a2014c9c28180"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "6f17531967d94ecaf489483ae002a428"
server_key_iv: "983cb8b900573e228d45e3e2b3806909"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.541257 2902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "b8d5e4c73750483ca02a2014c9c28180"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "6f17531967d94ecaf489483ae002a428"
server_key_iv: "983cb8b900573e228d45e3e2b3806909"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.542582 2902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:33.543398 2917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.543587 2902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:33.543674 2902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "b8d5e4c73750483ca02a2014c9c28180"
format_stamp: "Formatted at 2026-06-19 12:09:33 on dist-test-slave-c910"
server_key: "6f17531967d94ecaf489483ae002a428"
server_key_iv: "983cb8b900573e228d45e3e2b3806909"
server_key_version: "encryptionkey@0"
I20260619 12:09:33.543762 2902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:33.561836 2902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:33.563979 2902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:33.564141 2902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:33.564661 2902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:33.564730 2902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.564764 2902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:33.564795 2902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.571655 2902 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:45719
I20260619 12:09:33.571740 3032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:45719 every 8 connection(s)
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:33.572170 2902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:33.575466 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.565143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:59317 (local address 127.0.201.62:44317)
0619 12:09:33.565217 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.565221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.565429 (+ 208us) server_negotiation.cc:407] Connection header received
0619 12:09:33.565909 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.565911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.565926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.566008 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.566863 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.567704 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.569339 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.569679 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.569752 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.572581 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.572583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.572583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.572609 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.574296 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.574656 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.574657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.574657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.574707 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.574889 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.574890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.574890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.574990 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.575051 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.575340 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.575403 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:09:33.575678 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.565111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.201.3:59317)
0619 12:09:33.565297 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.565338 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.565810 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.566112 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.566116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.566462 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.566754 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.566761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.567824 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.567825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.569230 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.569232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.569772 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.569772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.569873 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.570438 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.570460 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.572469 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.574369 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.574374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.574376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.574592 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.574747 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.574747 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.574747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.574829 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.575055 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.575059 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.575173 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.575313 (+ 140us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.575435 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":121,"threads_started":1}
I20260619 12:09:33.576179 3033 heartbeater.cc:347] Connected to a master server at 127.0.201.62:44317
I20260619 12:09:33.576259 3033 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:33.576439 3033 heartbeater.cc:511] Master 127.0.201.62:44317 requested a full tablet report, sending...
I20260619 12:09:33.576826 2550 ts_manager.cc:194] Registered new tserver with Master: b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719)
I20260619 12:09:33.577275 2550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:59317
I20260619 12:09:33.581756 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 2902
I20260619 12:09:33.581859 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:33.582069 804 external_mini_cluster.cc:1428] Setting key 453d79334df364e0dea36210ca288e02
I20260619 12:09:33.582520 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.577244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.201.3:45965)
0619 12:09:33.577388 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.577399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.577452 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.577580 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.577581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.577785 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.577862 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.577864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.578457 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.578458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.578953 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.578954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.579280 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.579281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.579353 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.579796 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.579815 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.580081 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.581470 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.581471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.581472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.581634 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.581857 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.581858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.581858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.581922 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.582171 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.582173 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.582223 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.582299 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.582407 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260619 12:09:33.582551 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.577291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:45965 (local address 127.0.201.62:44317)
0619 12:09:33.577332 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.577333 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.577458 (+ 125us) server_negotiation.cc:407] Connection header received
0619 12:09:33.577516 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.577518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.577528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.577578 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.577934 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.578375 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.579018 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.579185 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.579283 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.580185 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.580186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.580186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.580191 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.581388 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.581709 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.581710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.581710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.581770 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.581974 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.581975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.581975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.582099 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.582169 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.582397 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.582464 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:33.583140 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:33.605628 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.598726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.0.201.62:44317)
0619 12:09:33.598787 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.598789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.598798 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:33.598865 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.598867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.598879 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.598940 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.599552 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.600044 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.600754 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.600947 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.601019 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.603338 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.603340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.603340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.603347 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.604547 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.604936 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.604937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.604938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.604996 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.605173 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.605173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.605174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.605279 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.605372 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.605438 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.605502 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:09:33.607362 2549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:33.608919 2549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:33.615340 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.611363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.0.201.3:45719)
0619 12:09:33.611751 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.611755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.611764 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:33.611796 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.611798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.611812 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.611902 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:33.612073 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.612865 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.614230 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.614870 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.614955 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.615018 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.615183 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.615248 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":78,"threads_started":1}
I20260619 12:09:33.615340 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.611269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.0.1:36284)
0619 12:09:33.611441 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.611449 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.611503 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.611909 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.611912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.611915 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:33.611996 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.611998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.612955 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.612956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.614131 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.614133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.614954 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.614955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.615052 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.615067 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.615165 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.615254 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
I20260619 12:09:33.615553 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.611264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38315 (local address 127.0.0.1:49532)
0619 12:09:33.611374 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.611417 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.611564 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.612032 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.612033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.612035 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:33.612078 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.612079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.613427 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.613428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.614295 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.614297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.615309 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.615310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.615373 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.615377 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.615408 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.615448 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
I20260619 12:09:33.615706 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.611472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:42943 (local address 127.0.0.1:57266)
0619 12:09:33.611604 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.611613 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.611646 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.612078 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.612078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.612080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:33.612113 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.612114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.613397 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.613398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.614460 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.614462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.615433 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.615434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.615480 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.615483 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.615538 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.615572 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260619 12:09:33.615705 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.611575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.0.201.2:42943)
0619 12:09:33.611734 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.611739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.611750 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:33.611789 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.611791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.611807 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.611902 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:33.612188 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.613306 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.614536 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.615291 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.615347 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.615403 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.615571 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.615617 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260619 12:09:33.615904 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.611679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.0.201.1:38315)
0619 12:09:33.611808 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.611812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.611822 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:33.611886 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.611888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.611903 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.612152 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:33.612213 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.613327 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.614387 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.615182 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.615668 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.615735 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.615764 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.615809 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260619 12:09:33.616585 2827 tablet_service.cc:1467] Processing CreateTablet for tablet d2e41f10e6b54516a104f08a5bed26b6 (DEFAULT_TABLE table=test-table [id=885930c87357474a9c7d0680ead0ec7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:33.616596 2687 tablet_service.cc:1467] Processing CreateTablet for tablet d2e41f10e6b54516a104f08a5bed26b6 (DEFAULT_TABLE table=test-table [id=885930c87357474a9c7d0680ead0ec7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:33.616919 2687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2e41f10e6b54516a104f08a5bed26b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.616920 2827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2e41f10e6b54516a104f08a5bed26b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.619975 3056 tablet_bootstrap.cc:492] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Bootstrap starting.
I20260619 12:09:33.620007 3057 tablet_bootstrap.cc:492] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Bootstrap starting.
I20260619 12:09:33.620586 3057 tablet_bootstrap.cc:654] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.620487 2967 tablet_service.cc:1467] Processing CreateTablet for tablet d2e41f10e6b54516a104f08a5bed26b6 (DEFAULT_TABLE table=test-table [id=885930c87357474a9c7d0680ead0ec7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:33.620671 3056 tablet_bootstrap.cc:654] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.620815 2967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2e41f10e6b54516a104f08a5bed26b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.620937 3057 log.cc:824] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:33.620963 3056 log.cc:824] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:33.621610 3057 tablet_bootstrap.cc:492] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: No bootstrap required, opened a new log
I20260619 12:09:33.621691 3057 ts_tablet_manager.cc:1397] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:33.623384 3060 tablet_bootstrap.cc:492] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Bootstrap starting.
I20260619 12:09:33.623328 3057 raft_consensus.cc:348] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.623500 3057 raft_consensus.cc:374] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.623545 3057 raft_consensus.cc:729] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.623660 3057 consensus_queue.cc:260] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.623942 3056 tablet_bootstrap.cc:492] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: No bootstrap required, opened a new log
I20260619 12:09:33.623945 3057 ts_tablet_manager.cc:1428] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:33.623957 2893 heartbeater.cc:503] Master 127.0.201.62:44317 was elected leader, sending a full tablet report...
I20260619 12:09:33.624008 3056 ts_tablet_manager.cc:1397] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:09:33.624483 3060 tablet_bootstrap.cc:654] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.624800 3060 log.cc:824] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:33.625597 3060 tablet_bootstrap.cc:492] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: No bootstrap required, opened a new log
I20260619 12:09:33.625658 3060 ts_tablet_manager.cc:1397] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:33.625980 3056 raft_consensus.cc:348] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.626101 3056 raft_consensus.cc:374] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.626122 3056 raft_consensus.cc:729] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e95e347c604116a8c3e31cded4f683, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.626236 3056 consensus_queue.cc:260] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.626502 3056 ts_tablet_manager.cc:1428] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:09:33.626533 2753 heartbeater.cc:503] Master 127.0.201.62:44317 was elected leader, sending a full tablet report...
I20260619 12:09:33.627544 3060 raft_consensus.cc:348] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.627657 3060 raft_consensus.cc:374] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.627677 3060 raft_consensus.cc:729] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.627776 3060 consensus_queue.cc:260] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.627988 3060 ts_tablet_manager.cc:1428] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:09:33.628041 3033 heartbeater.cc:503] Master 127.0.201.62:44317 was elected leader, sending a full tablet report...
W20260619 12:09:33.664758 2894 tablet.cc:2369] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:09:33.757813 2754 tablet.cc:2369] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:09:33.822829 3034 tablet.cc:2369] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:33.904811 3061 raft_consensus.cc:482] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:33.904939 3061 raft_consensus.cc:504] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.905369 3061 leader_election.cc:288] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719), 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315)
I20260619 12:09:33.908236 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.905638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:47427 (local address 127.0.201.3:45719)
0619 12:09:33.905685 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.905687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.905700 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:09:33.905738 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.905739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.905759 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.905818 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:33.905991 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.906501 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.907487 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.907879 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.907933 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.907941 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.908122 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.908162 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:09:33.908254 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.905658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:57409 (local address 127.0.201.1:38315)
0619 12:09:33.905698 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.905700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.905788 (+ 88us) server_negotiation.cc:407] Connection header received
0619 12:09:33.905835 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.905836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.905847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.905907 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:33.906092 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.906529 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.907552 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.907891 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.907981 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.907991 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.908149 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.908189 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:33.908253 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.905567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.201.2:47427)
0619 12:09:33.905627 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.905639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.905686 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.905832 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.905833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.905835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:33.905910 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.905912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.906581 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.906581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.907404 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.907406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.907962 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.907963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.908048 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.908051 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.908092 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.908144 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":24}
I20260619 12:09:33.908272 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.905581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38315 (local address 127.0.201.2:57409)
0619 12:09:33.905706 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.905716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.905778 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.905926 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.905927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.905928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:33.906015 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.906016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.906642 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.906643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.907454 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.907457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.907995 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.907996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.908059 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.908062 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.908136 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.908171 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260619 12:09:33.908627 2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e41f10e6b54516a104f08a5bed26b6" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d5e4c73750483ca02a2014c9c28180" is_pre_election: true
I20260619 12:09:33.908798 2987 raft_consensus.cc:2454] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 0.
I20260619 12:09:33.908854 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e41f10e6b54516a104f08a5bed26b6" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683" is_pre_election: true
I20260619 12:09:33.909019 2707 raft_consensus.cc:2454] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 0.
I20260619 12:09:33.909103 2778 leader_election.cc:302] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:33.909225 3061 raft_consensus.cc:2781] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:33.909284 3061 raft_consensus.cc:482] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:33.909320 3061 raft_consensus.cc:3037] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:33.910068 3061 raft_consensus.cc:504] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.910204 3061 leader_election.cc:288] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 election: Requested vote from peers b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719), 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315)
I20260619 12:09:33.910357 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e41f10e6b54516a104f08a5bed26b6" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683"
I20260619 12:09:33.910353 2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e41f10e6b54516a104f08a5bed26b6" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d5e4c73750483ca02a2014c9c28180"
I20260619 12:09:33.910442 2987 raft_consensus.cc:3037] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:33.910447 2707 raft_consensus.cc:3037] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:33.911161 2987 raft_consensus.cc:2454] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 1.
I20260619 12:09:33.911172 2707 raft_consensus.cc:2454] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 1.
I20260619 12:09:33.911357 2779 leader_election.cc:302] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, 90e95e347c604116a8c3e31cded4f683; no voters:
I20260619 12:09:33.911463 3061 raft_consensus.cc:2781] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:33.911649 3061 raft_consensus.cc:686] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 1 LEADER]: Becoming Leader. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Running, Role: LEADER
I20260619 12:09:33.911752 3061 consensus_queue.cc:237] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } }
I20260619 12:09:33.912602 2549 catalog_manager.cc:5495] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc reported cstate change: term changed from 0 to 1, leader changed from <none> to 73aa358785544f6b86db4527961a7adc (127.0.201.2). New cstate: current_term: 1 leader_uuid: "73aa358785544f6b86db4527961a7adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } health_report { overall_health: HEALTHY } } }
Jun 19 12:09:33 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:33.990563 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.983959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.0.201.62:44317)
0619 12:09:33.984080 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:33.984082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:33.984139 (+ 57us) server_negotiation.cc:407] Connection header received
0619 12:09:33.984186 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:33.984186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:33.984196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:33.984255 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:33.984779 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.985263 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.986039 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.986220 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.986273 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.988329 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:33.988330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:33.988330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:33.988335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:33.989645 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.989960 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.989960 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.989961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.990013 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:33.990167 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:33.990167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:33.990167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:33.990267 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:33.990317 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:33.990440 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 12:09:33.990500 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 12:09:33.990643 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:33.983875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44317 (local address 127.0.0.1:44374)
0619 12:09:33.984046 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:33.984071 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:33.984142 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:33.984296 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:33.984297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:33.984650 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:33.984699 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.984701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.985331 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.985357 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.985964 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:33.985966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:33.986283 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:33.986284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:33.986323 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:33.986893 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:33.986902 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:33.988258 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:33.989715 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.989721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.989723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.989902 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.990067 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:33.990068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:33.990068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:33.990120 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:33.990319 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:33.990324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:33.990368 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 12:09:33.990423 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:09:33.990568 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":60,"thread_start_us":61,"threads_started":1}
I20260619 12:09:33.992559 2549 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:44374:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:09:33.992776 2549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:33.994392 2967 tablet_service.cc:1467] Processing CreateTablet for tablet 4028d80797384d2180ba94d07a62864b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6053af52abaf43b78b59078de8269aff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:33.994413 2687 tablet_service.cc:1467] Processing CreateTablet for tablet 4028d80797384d2180ba94d07a62864b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6053af52abaf43b78b59078de8269aff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:33.994429 2827 tablet_service.cc:1467] Processing CreateTablet for tablet 4028d80797384d2180ba94d07a62864b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6053af52abaf43b78b59078de8269aff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:33.994566 2967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4028d80797384d2180ba94d07a62864b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.994571 2687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4028d80797384d2180ba94d07a62864b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.994570 2827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4028d80797384d2180ba94d07a62864b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:33.995854 3060 tablet_bootstrap.cc:492] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: Bootstrap starting.
I20260619 12:09:33.996116 3056 tablet_bootstrap.cc:492] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: Bootstrap starting.
I20260619 12:09:33.996132 3057 tablet_bootstrap.cc:492] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: Bootstrap starting.
I20260619 12:09:33.996479 3060 tablet_bootstrap.cc:654] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.996502 3056 tablet_bootstrap.cc:654] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.996502 3057 tablet_bootstrap.cc:654] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:33.997202 3060 tablet_bootstrap.cc:492] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: No bootstrap required, opened a new log
I20260619 12:09:33.997257 3060 ts_tablet_manager.cc:1397] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:33.997385 3056 tablet_bootstrap.cc:492] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: No bootstrap required, opened a new log
I20260619 12:09:33.997437 3056 ts_tablet_manager.cc:1397] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:33.997450 3060 raft_consensus.cc:348] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:33.997548 3060 raft_consensus.cc:374] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.997582 3060 raft_consensus.cc:729] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.997572 3056 raft_consensus.cc:348] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:33.997634 3056 raft_consensus.cc:374] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.997655 3056 raft_consensus.cc:729] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e95e347c604116a8c3e31cded4f683, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.997645 3060 consensus_queue.cc:260] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:33.997699 3056 consensus_queue.cc:260] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:33.997748 3060 ts_tablet_manager.cc:1428] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:33.997783 3056 ts_tablet_manager.cc:1428] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.997818 3064 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:33.997840 3063 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:33.997889 3064 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:33.997918 3063 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:33.997982 3057 tablet_bootstrap.cc:492] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: No bootstrap required, opened a new log
I20260619 12:09:33.998023 3057 ts_tablet_manager.cc:1397] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:33.998157 3057 raft_consensus.cc:348] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:33.998252 3057 raft_consensus.cc:374] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:33.998287 3057 raft_consensus.cc:729] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Initialized, Role: FOLLOWER
I20260619 12:09:33.998335 3057 consensus_queue.cc:260] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:33.998420 3057 ts_tablet_manager.cc:1428] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:33.998616 3061 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:33.998718 3061 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:34.010949 3064 raft_consensus.cc:482] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:34.011037 3064 raft_consensus.cc:504] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.011461 3064 leader_election.cc:288] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315), 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943)
I20260619 12:09:34.014156 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.011656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:40127 (local address 127.0.201.1:38315)
0619 12:09:34.011703 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.011705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.011717 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:09:34.011793 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.011794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.011804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.011861 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:34.012048 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.012530 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.013439 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.013802 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.013885 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.013896 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.014058 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.014091 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:34.014160 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.011603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38315 (local address 127.0.201.3:40127)
0619 12:09:34.011666 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.011680 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.011733 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.011900 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.011901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.011903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:34.011969 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.011971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.012609 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.012610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.013346 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.013348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.013885 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.013886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.013965 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.013968 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.014050 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.014090 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 12:09:34.014391 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4028d80797384d2180ba94d07a62864b" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683" is_pre_election: true
I20260619 12:09:34.014487 2707 raft_consensus.cc:2454] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 0.
I20260619 12:09:34.014492 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.011780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:58787 (local address 127.0.201.2:42943)
0619 12:09:34.011829 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.011830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.011945 (+ 115us) server_negotiation.cc:407] Connection header received
0619 12:09:34.012041 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.012042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.012087 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.012137 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:34.012287 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.012751 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.013637 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.013994 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.014073 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.014083 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.014383 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.014430 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:09:34.014516 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.011706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:42943 (local address 127.0.201.3:58787)
0619 12:09:34.011883 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.011891 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.011958 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.012139 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.012140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.012141 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:34.012220 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.012221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.012828 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.012830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.013548 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.013550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.014093 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.014094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.014160 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.014163 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.014369 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.014406 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260619 12:09:34.014688 2919 leader_election.cc:302] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e95e347c604116a8c3e31cded4f683, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:34.014811 3064 raft_consensus.cc:2781] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:34.014856 3064 raft_consensus.cc:482] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:34.014882 3064 raft_consensus.cc:3037] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:34.015017 2847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4028d80797384d2180ba94d07a62864b" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73aa358785544f6b86db4527961a7adc" is_pre_election: true
I20260619 12:09:34.015113 2847 raft_consensus.cc:2454] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 0.
I20260619 12:09:34.015779 3064 raft_consensus.cc:504] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.015918 3064 leader_election.cc:288] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 election: Requested vote from peers 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315), 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943)
I20260619 12:09:34.016089 2847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4028d80797384d2180ba94d07a62864b" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73aa358785544f6b86db4527961a7adc"
I20260619 12:09:34.016098 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4028d80797384d2180ba94d07a62864b" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683"
I20260619 12:09:34.016166 2707 raft_consensus.cc:3037] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:34.016171 2847 raft_consensus.cc:3037] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:34.016742 2707 raft_consensus.cc:2454] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 1.
I20260619 12:09:34.016862 2847 raft_consensus.cc:2454] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 1.
I20260619 12:09:34.016914 2919 leader_election.cc:302] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e95e347c604116a8c3e31cded4f683, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:34.016999 3064 raft_consensus.cc:2781] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:34.017132 3064 raft_consensus.cc:686] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [term 1 LEADER]: Becoming Leader. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Running, Role: LEADER
I20260619 12:09:34.017227 3064 consensus_queue.cc:237] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.017691 3078 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: TxnStatusTablet state changed. Reason: New leader b8d5e4c73750483ca02a2014c9c28180. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.017771 3078 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:34.017982 2549 catalog_manager.cc:5495] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } health_report { overall_health: HEALTHY } } }
I20260619 12:09:34.018249 3080 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:09:34.018375 3080 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:09:34.026904 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.024228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.0.1:36290)
0619 12:09:34.024355 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.024366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.024426 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.024641 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.024642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.024643 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:34.024689 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.024691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.025400 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.025401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.025978 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.025979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.026379 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.026380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.026443 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.026462 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.026740 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.026747 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.026792 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.026837 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 12:09:34.026893 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.024284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.0.201.3:45719)
0619 12:09:34.024322 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.024324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.024419 (+ 95us) server_negotiation.cc:407] Connection header received
0619 12:09:34.024503 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.024503 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.024536 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.024616 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.024766 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.025319 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.026088 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.026294 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.026364 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.026560 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.026665 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.026747 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.026793 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.026814 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:09:34.028853 2847 raft_consensus.cc:1261] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Refusing update from remote peer b8d5e4c73750483ca02a2014c9c28180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:34.028891 2707 raft_consensus.cc:1261] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Refusing update from remote peer b8d5e4c73750483ca02a2014c9c28180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:34.029151 3078 consensus_queue.cc:1035] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:34.029282 3064 consensus_queue.cc:1035] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:34.030319 3061 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: TxnStatusTablet state changed. Reason: New leader b8d5e4c73750483ca02a2014c9c28180. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.030400 3061 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:34.030440 3063 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: TxnStatusTablet state changed. Reason: New leader b8d5e4c73750483ca02a2014c9c28180. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.030525 3063 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:34.031277 3078 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.031364 3078 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:34.031330 3061 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.031409 3061 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P 73aa358785544f6b86db4527961a7adc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:34.032069 3089 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.032148 3089 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P b8d5e4c73750483ca02a2014c9c28180: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:34.032284 3085 mvcc.cc:205] Tried to move back new op lower bound from 7298543509619728384 to 7298543509575102464. Current Snapshot: MvccSnapshot[applied={T|T < 7298543509619728384}]
I20260619 12:09:34.032351 3063 tablet_replica.cc:439] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } } }
I20260619 12:09:34.032423 3063 tablet_replica.cc:442] T 4028d80797384d2180ba94d07a62864b P 90e95e347c604116a8c3e31cded4f683: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:34.033303 3084 mvcc.cc:205] Tried to move back new op lower bound from 7298543509619728384 to 7298543509575102464. Current Snapshot: MvccSnapshot[applied={T|T < 7298543509619728384 or (T in {7298543509619728384})}]
I20260619 12:09:34.036559 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.034164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57272 (local address 127.0.201.2:42943)
0619 12:09:34.034205 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.034206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.034215 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:34.034280 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.034281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.034297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.034370 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.034465 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.034929 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.035774 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.035989 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.036073 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.036232 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.036331 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.036417 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.036469 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.036488 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:34.041882 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.039068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:58723 (local address 127.0.201.3:45719)
0619 12:09:34.039109 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.039111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.039255 (+ 144us) server_negotiation.cc:407] Connection header received
0619 12:09:34.039277 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.039278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.039287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.039340 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.039530 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.039990 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.041163 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.041348 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.041435 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.041654 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.041683 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.041738 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.041812 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.041828 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:34.041953 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.038969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.201.2:58723)
0619 12:09:34.039113 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.039123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.039214 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.039349 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.039350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.039351 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:34.039413 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.039414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.040068 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.040069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.041065 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.041067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.041444 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.041444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.041509 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.041522 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.041754 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.041756 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.041810 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.041853 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260619 12:09:34.044577 2707 raft_consensus.cc:1261] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Refusing update from remote peer 73aa358785544f6b86db4527961a7adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:34.044700 2987 raft_consensus.cc:1261] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Refusing update from remote peer 73aa358785544f6b86db4527961a7adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:34.044798 3061 consensus_queue.cc:1035] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:34.044922 3066 consensus_queue.cc:1035] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:34.046687 3083 mvcc.cc:205] Tried to move back new op lower bound from 7298543509684969472 to 7298543509143293952. Current Snapshot: MvccSnapshot[applied={T|T < 7298543509684969472}]
I20260619 12:09:34.053129 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.050736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:42943 (local address 127.0.201.3:51689)
0619 12:09:34.050805 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.050817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.050880 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.051005 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.051005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.051006 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:34.051064 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.051065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.051748 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.051749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.052384 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.052387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.052731 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.052732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.052797 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.052812 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.052977 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.052979 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.053029 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.053066 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:09:34.053117 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.050799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:51689 (local address 127.0.201.2:42943)
0619 12:09:34.050835 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.050837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.050864 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:09:34.050937 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.050937 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.050948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.051002 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.051123 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.051657 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.052446 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.052648 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.052730 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.052892 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.052924 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.052976 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.053033 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.053049 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:34.062415 2549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:34.062536 2549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:34.063256 2549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Sending DeleteTablet for 3 replicas of tablet d2e41f10e6b54516a104f08a5bed26b6
I20260619 12:09:34.063724 2967 tablet_service.cc:1514] Processing DeleteTablet for tablet d2e41f10e6b54516a104f08a5bed26b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:34 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36284
I20260619 12:09:34.063947 2687 tablet_service.cc:1514] Processing DeleteTablet for tablet d2e41f10e6b54516a104f08a5bed26b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:34 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49532
I20260619 12:09:34.064218 2827 tablet_service.cc:1514] Processing DeleteTablet for tablet d2e41f10e6b54516a104f08a5bed26b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:34 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:57266
I20260619 12:09:34.064419 2549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:34.064544 2549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:34.065778 2967 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee2b373f6434ffa83c323ac64cb480c (DEFAULT_TABLE table=test-table [id=0fd6e15f1c9a44d5badd2a5f872677f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:34.065896 2967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee2b373f6434ffa83c323ac64cb480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:34.067214 3060 tablet_bootstrap.cc:492] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Bootstrap starting.
I20260619 12:09:34.067637 3060 tablet_bootstrap.cc:654] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:34.068156 2687 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee2b373f6434ffa83c323ac64cb480c (DEFAULT_TABLE table=test-table [id=0fd6e15f1c9a44d5badd2a5f872677f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:34.068291 2687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee2b373f6434ffa83c323ac64cb480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:34.069475 3060 tablet_bootstrap.cc:492] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: No bootstrap required, opened a new log
I20260619 12:09:34.069519 3060 ts_tablet_manager.cc:1397] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:34.069665 3060 raft_consensus.cc:348] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.069736 3060 raft_consensus.cc:374] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:34.069757 3060 raft_consensus.cc:729] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Initialized, Role: FOLLOWER
I20260619 12:09:34.069784 3056 tablet_bootstrap.cc:492] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Bootstrap starting.
I20260619 12:09:34.069801 3060 consensus_queue.cc:260] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.069928 3060 ts_tablet_manager.cc:1428] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:34.070037 2827 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee2b373f6434ffa83c323ac64cb480c (DEFAULT_TABLE table=test-table [id=0fd6e15f1c9a44d5badd2a5f872677f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:34.070145 2827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee2b373f6434ffa83c323ac64cb480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:34.070182 3056 tablet_bootstrap.cc:654] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:34.070551 3103 tablet_replica.cc:330] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: stopping tablet replica
I20260619 12:09:34.070663 3103 raft_consensus.cc:2229] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:34.070729 3103 raft_consensus.cc:2258] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:34.071465 3057 tablet_bootstrap.cc:492] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Bootstrap starting.
I20260619 12:09:34.071642 3104 tablet_replica.cc:330] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: stopping tablet replica
I20260619 12:09:34.071765 3104 raft_consensus.cc:2229] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:34.071748 3102 tablet_replica.cc:330] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: stopping tablet replica
I20260619 12:09:34.071890 3104 raft_consensus.cc:2258] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:34.071906 3057 tablet_bootstrap.cc:654] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:34.072016 3102 raft_consensus.cc:2229] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:34.072135 3102 raft_consensus.cc:2258] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:34.072415 3103 ts_tablet_manager.cc:1905] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:34.072436 3104 ts_tablet_manager.cc:1905] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:34.072638 3102 ts_tablet_manager.cc:1905] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:34.073939 3103 ts_tablet_manager.cc:1918] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:34.073957 3104 ts_tablet_manager.cc:1918] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:34.074023 3103 log.cc:1196] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/d2e41f10e6b54516a104f08a5bed26b6
I20260619 12:09:34.074023 3104 log.cc:1196] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/d2e41f10e6b54516a104f08a5bed26b6
I20260619 12:09:34.074191 3102 ts_tablet_manager.cc:1918] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:34.074285 3103 ts_tablet_manager.cc:1939] T d2e41f10e6b54516a104f08a5bed26b6 P 90e95e347c604116a8c3e31cded4f683: Deleting consensus metadata
I20260619 12:09:34.074291 3104 ts_tablet_manager.cc:1939] T d2e41f10e6b54516a104f08a5bed26b6 P 73aa358785544f6b86db4527961a7adc: Deleting consensus metadata
I20260619 12:09:34.074283 3102 log.cc:1196] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/d2e41f10e6b54516a104f08a5bed26b6
I20260619 12:09:34.074548 3102 ts_tablet_manager.cc:1939] T d2e41f10e6b54516a104f08a5bed26b6 P b8d5e4c73750483ca02a2014c9c28180: Deleting consensus metadata
I20260619 12:09:34.074685 2535 catalog_manager.cc:4841] TS 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943): tablet d2e41f10e6b54516a104f08a5bed26b6 (table test-table [id=885930c87357474a9c7d0680ead0ec7f]) successfully deleted
I20260619 12:09:34.074777 2535 catalog_manager.cc:4841] TS 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315): tablet d2e41f10e6b54516a104f08a5bed26b6 (table test-table [id=885930c87357474a9c7d0680ead0ec7f]) successfully deleted
I20260619 12:09:34.075047 2534 catalog_manager.cc:4841] TS b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719): tablet d2e41f10e6b54516a104f08a5bed26b6 (table test-table [id=885930c87357474a9c7d0680ead0ec7f]) successfully deleted
I20260619 12:09:34.075553 3056 tablet_bootstrap.cc:492] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: No bootstrap required, opened a new log
I20260619 12:09:34.075608 3056 ts_tablet_manager.cc:1397] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 12:09:34.075671 3057 tablet_bootstrap.cc:492] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: No bootstrap required, opened a new log
I20260619 12:09:34.075721 3057 ts_tablet_manager.cc:1397] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:09:34.075731 3056 raft_consensus.cc:348] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.075809 3056 raft_consensus.cc:374] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:34.075830 3056 raft_consensus.cc:729] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e95e347c604116a8c3e31cded4f683, State: Initialized, Role: FOLLOWER
I20260619 12:09:34.075876 3056 consensus_queue.cc:260] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.075898 3057 raft_consensus.cc:348] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.075963 3056 ts_tablet_manager.cc:1428] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:34.075968 3057 raft_consensus.cc:374] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:34.075994 3057 raft_consensus.cc:729] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Initialized, Role: FOLLOWER
I20260619 12:09:34.076041 3057 consensus_queue.cc:260] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.076125 3057 ts_tablet_manager.cc:1428] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:34.276394 3063 raft_consensus.cc:482] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:34.276513 3063 raft_consensus.cc:504] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.276921 3063 leader_election.cc:288] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943), b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719)
I20260619 12:09:34.280848 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.277192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:48235 (local address 127.0.201.2:42943)
0619 12:09:34.277320 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.277322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.277351 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:09:34.277425 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.277426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.277435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.277488 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:34.277643 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.278398 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.279947 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.280517 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.280578 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.280587 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.280745 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.280777 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:09:34.280848 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.277196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:55351 (local address 127.0.201.3:45719)
0619 12:09:34.277247 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.277250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.277362 (+ 112us) server_negotiation.cc:407] Connection header received
0619 12:09:34.277416 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.277417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.277428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.277488 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:34.277642 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.278393 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.279923 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.280490 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.280571 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.280582 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.280736 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.280791 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:09:34.280882 3108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.277136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.201.1:55351)
0619 12:09:34.277284 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.277295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.277354 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.277506 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.277507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.277511 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:34.277575 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.277576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.278474 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.278475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.279830 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.279834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.280592 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.280593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.280658 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.280663 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.280722 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.280774 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":92,"thread_start_us":81,"threads_started":1}
I20260619 12:09:34.280889 3109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.277133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:42943 (local address 127.0.201.1:48235)
0619 12:09:34.277282 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.277295 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.277354 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.277506 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.277507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.277509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:34.277569 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.277571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.278515 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.278516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.279830 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.279834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.280593 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.280594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.280658 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.280664 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.280722 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.280773 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":34,"thread_start_us":64,"threads_started":1}
I20260619 12:09:34.281126 2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee2b373f6434ffa83c323ac64cb480c" candidate_uuid: "90e95e347c604116a8c3e31cded4f683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d5e4c73750483ca02a2014c9c28180" is_pre_election: true
I20260619 12:09:34.281211 2987 raft_consensus.cc:2454] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e95e347c604116a8c3e31cded4f683 in term 0.
I20260619 12:09:34.281309 2847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee2b373f6434ffa83c323ac64cb480c" candidate_uuid: "90e95e347c604116a8c3e31cded4f683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73aa358785544f6b86db4527961a7adc" is_pre_election: true
I20260619 12:09:34.281401 2847 raft_consensus.cc:2454] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e95e347c604116a8c3e31cded4f683 in term 0.
I20260619 12:09:34.281584 2639 leader_election.cc:302] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, 90e95e347c604116a8c3e31cded4f683; no voters:
I20260619 12:09:34.281716 3063 raft_consensus.cc:2781] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:34.281775 3063 raft_consensus.cc:482] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:34.281791 3063 raft_consensus.cc:3037] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:34.282421 3063 raft_consensus.cc:504] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.282565 3063 leader_election.cc:288] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [CANDIDATE]: Term 1 election: Requested vote from peers 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943), b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719)
I20260619 12:09:34.282683 2847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee2b373f6434ffa83c323ac64cb480c" candidate_uuid: "90e95e347c604116a8c3e31cded4f683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73aa358785544f6b86db4527961a7adc"
I20260619 12:09:34.282707 2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee2b373f6434ffa83c323ac64cb480c" candidate_uuid: "90e95e347c604116a8c3e31cded4f683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d5e4c73750483ca02a2014c9c28180"
I20260619 12:09:34.282770 2847 raft_consensus.cc:3037] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:34.282804 2987 raft_consensus.cc:3037] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:34.283387 2847 raft_consensus.cc:2454] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e95e347c604116a8c3e31cded4f683 in term 1.
I20260619 12:09:34.283543 2987 raft_consensus.cc:2454] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e95e347c604116a8c3e31cded4f683 in term 1.
I20260619 12:09:34.283537 2639 leader_election.cc:302] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, 90e95e347c604116a8c3e31cded4f683; no voters:
I20260619 12:09:34.283648 3063 raft_consensus.cc:2781] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:34.283818 3063 raft_consensus.cc:686] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 1 LEADER]: Becoming Leader. State: Replica: 90e95e347c604116a8c3e31cded4f683, State: Running, Role: LEADER
I20260619 12:09:34.283890 3063 consensus_queue.cc:237] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:34.284586 2550 catalog_manager.cc:5495] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90e95e347c604116a8c3e31cded4f683 (127.0.201.1). New cstate: current_term: 1 leader_uuid: "90e95e347c604116a8c3e31cded4f683" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:34.302799 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.300611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.0.201.1:38315)
0619 12:09:34.300668 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.300670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.300677 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:34.300708 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.300709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.300725 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.300785 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.300910 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.301369 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.302087 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.302256 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.302304 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.302513 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.302619 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.302688 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.302723 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.302743 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:09:34.307864 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.305174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:41053 (local address 127.0.201.3:45719)
0619 12:09:34.305215 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.305217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.305369 (+ 152us) server_negotiation.cc:407] Connection header received
0619 12:09:34.305421 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.305421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.305430 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.305471 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.305870 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.306306 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.307073 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.307320 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.307418 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.307668 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.307694 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.307732 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.307791 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.307805 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:34.307888 3114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.305115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.201.1:41053)
0619 12:09:34.305288 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.305297 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.305379 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.305481 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.305481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.305482 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:34.305786 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.305788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.306407 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.306407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.306999 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.307000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.307521 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.307522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.307579 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.307591 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.307747 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.307748 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.307784 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.307820 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260619 12:09:34.309993 2847 raft_consensus.cc:1261] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Refusing update from remote peer 90e95e347c604116a8c3e31cded4f683: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:34.310026 2987 raft_consensus.cc:1261] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Refusing update from remote peer 90e95e347c604116a8c3e31cded4f683: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:34.310248 3110 consensus_queue.cc:1035] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:34.310403 3063 consensus_queue.cc:1035] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:34.317397 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.314985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:47683 (local address 127.0.201.1:38315)
0619 12:09:34.315041 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:34.315043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:34.315050 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:34.315070 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:34.315071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:34.315080 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:34.315126 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:34.315303 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.315744 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.316640 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.316933 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.317017 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.317158 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.317190 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.317231 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:34.317284 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:09:34.317302 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:09:34.317402 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:34.314840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38315 (local address 127.0.201.3:47683)
0619 12:09:34.314880 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:34.314891 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:34.314929 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:34.315170 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:34.315171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:34.315172 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:34.315247 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.315248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.315825 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.315826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.316563 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:34.316564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:34.317017 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:34.317017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:34.317069 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:34.317073 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:34.317241 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:34.317243 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:34.317284 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 12:09:34.317340 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 12:09:34.317675 3084 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 12:09:35.840829 2550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:35.840929 2550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:35.841673 2550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Sending DeleteTablet for 3 replicas of tablet 0ee2b373f6434ffa83c323ac64cb480c
I20260619 12:09:35.841971 2967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee2b373f6434ffa83c323ac64cb480c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:35 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36284
I20260619 12:09:35.841966 2827 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee2b373f6434ffa83c323ac64cb480c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:35 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:57266
I20260619 12:09:35.842142 2687 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee2b373f6434ffa83c323ac64cb480c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:35 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49532
I20260619 12:09:35.842162 3140 tablet_replica.cc:330] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: stopping tablet replica
I20260619 12:09:35.842226 3140 raft_consensus.cc:2229] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:35.842262 3140 raft_consensus.cc:2258] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:35.842319 3141 tablet_replica.cc:330] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: stopping tablet replica
I20260619 12:09:35.842368 3141 raft_consensus.cc:2229] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:35.842445 3141 raft_consensus.cc:2258] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:35.842518 3140 ts_tablet_manager.cc:1905] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:35.842659 3141 ts_tablet_manager.cc:1905] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:35.844470 3141 ts_tablet_manager.cc:1918] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:09:35.844537 3140 ts_tablet_manager.cc:1918] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:09:35.844583 3141 log.cc:1196] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/0ee2b373f6434ffa83c323ac64cb480c
I20260619 12:09:35.844592 3140 log.cc:1196] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/0ee2b373f6434ffa83c323ac64cb480c
I20260619 12:09:35.844810 3140 ts_tablet_manager.cc:1939] T 0ee2b373f6434ffa83c323ac64cb480c P b8d5e4c73750483ca02a2014c9c28180: Deleting consensus metadata
I20260619 12:09:35.844835 3141 ts_tablet_manager.cc:1939] T 0ee2b373f6434ffa83c323ac64cb480c P 90e95e347c604116a8c3e31cded4f683: Deleting consensus metadata
I20260619 12:09:35.845113 3139 tablet_replica.cc:330] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: stopping tablet replica
I20260619 12:09:35.845145 2534 catalog_manager.cc:4841] TS b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719): tablet 0ee2b373f6434ffa83c323ac64cb480c (table test-table [id=0fd6e15f1c9a44d5badd2a5f872677f5]) successfully deleted
I20260619 12:09:35.845124 2535 catalog_manager.cc:4841] TS 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315): tablet 0ee2b373f6434ffa83c323ac64cb480c (table test-table [id=0fd6e15f1c9a44d5badd2a5f872677f5]) successfully deleted
I20260619 12:09:35.845202 3139 raft_consensus.cc:2229] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:35.845239 3139 raft_consensus.cc:2258] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:35.845436 3139 ts_tablet_manager.cc:1905] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:35.846896 3139 ts_tablet_manager.cc:1918] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:09:35.846969 3139 log.cc:1196] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/0ee2b373f6434ffa83c323ac64cb480c
I20260619 12:09:35.847180 3139 ts_tablet_manager.cc:1939] T 0ee2b373f6434ffa83c323ac64cb480c P 73aa358785544f6b86db4527961a7adc: Deleting consensus metadata
I20260619 12:09:35.847548 2535 catalog_manager.cc:4841] TS 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943): tablet 0ee2b373f6434ffa83c323ac64cb480c (table test-table [id=0fd6e15f1c9a44d5badd2a5f872677f5]) successfully deleted
Jun 19 12:09:35 dist-test-slave-c910 krb5kdc[2505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:09:35 dist-test-slave-c910 krb5kdc[2505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:35.865597 3149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:35.858287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.0.201.62:44317)
0619 12:09:35.858413 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:35.858415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:35.858424 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:35.858488 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:35.858489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:35.858497 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:35.858556 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:35.859074 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:35.859692 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:35.860598 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:35.860790 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:35.860841 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:35.863065 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:35.863067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:35.863067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:35.863073 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:35.864528 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:35.864895 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:35.864895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:35.864896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:35.864949 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:35.865129 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:35.865130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:35.865130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:35.865224 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:35.865296 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:35.865425 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 12:09:35.865484 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260619 12:09:35.867305 2550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:35.867439 2550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:35.868851 2827 tablet_service.cc:1467] Processing CreateTablet for tablet ba6aaca0198b468fbd3b5b2e080efc56 (DEFAULT_TABLE table=test-table [id=7c49cc0a0c0840ce82003dafc560286b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:35.868844 2687 tablet_service.cc:1467] Processing CreateTablet for tablet ba6aaca0198b468fbd3b5b2e080efc56 (DEFAULT_TABLE table=test-table [id=7c49cc0a0c0840ce82003dafc560286b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:35.868916 2967 tablet_service.cc:1467] Processing CreateTablet for tablet ba6aaca0198b468fbd3b5b2e080efc56 (DEFAULT_TABLE table=test-table [id=7c49cc0a0c0840ce82003dafc560286b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:35.868993 2827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6aaca0198b468fbd3b5b2e080efc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:35.868989 2687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6aaca0198b468fbd3b5b2e080efc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:35.869042 2967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6aaca0198b468fbd3b5b2e080efc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:35.869920 3150 tablet_bootstrap.cc:492] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Bootstrap starting.
I20260619 12:09:35.870220 3150 tablet_bootstrap.cc:654] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:35.870518 3153 tablet_bootstrap.cc:492] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Bootstrap starting.
I20260619 12:09:35.870723 3150 tablet_bootstrap.cc:492] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: No bootstrap required, opened a new log
I20260619 12:09:35.870769 3150 ts_tablet_manager.cc:1397] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:35.870889 3153 tablet_bootstrap.cc:654] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:35.870885 3150 raft_consensus.cc:348] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.870958 3150 raft_consensus.cc:374] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:35.870990 3150 raft_consensus.cc:729] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Initialized, Role: FOLLOWER
I20260619 12:09:35.871047 3150 consensus_queue.cc:260] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.871217 3152 tablet_bootstrap.cc:492] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Bootstrap starting.
I20260619 12:09:35.871243 3150 ts_tablet_manager.cc:1428] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:35.871536 3153 tablet_bootstrap.cc:492] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: No bootstrap required, opened a new log
I20260619 12:09:35.871575 3153 ts_tablet_manager.cc:1397] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:35.871690 3153 raft_consensus.cc:348] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.871780 3153 raft_consensus.cc:374] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:35.871805 3153 raft_consensus.cc:729] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Initialized, Role: FOLLOWER
I20260619 12:09:35.871850 3153 consensus_queue.cc:260] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.871955 3153 ts_tablet_manager.cc:1428] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:35.872254 3152 tablet_bootstrap.cc:654] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:35.872843 3152 tablet_bootstrap.cc:492] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: No bootstrap required, opened a new log
I20260619 12:09:35.872897 3152 ts_tablet_manager.cc:1397] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:35.873008 3152 raft_consensus.cc:348] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.873090 3152 raft_consensus.cc:374] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:35.873121 3152 raft_consensus.cc:729] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e95e347c604116a8c3e31cded4f683, State: Initialized, Role: FOLLOWER
I20260619 12:09:35.873175 3152 consensus_queue.cc:260] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.873265 3152 ts_tablet_manager.cc:1428] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:35.987545 3155 raft_consensus.cc:482] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:35.987632 3155 raft_consensus.cc:504] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.987789 3155 leader_election.cc:288] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315), b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719)
I20260619 12:09:35.987977 2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6aaca0198b468fbd3b5b2e080efc56" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d5e4c73750483ca02a2014c9c28180" is_pre_election: true
I20260619 12:09:35.987959 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6aaca0198b468fbd3b5b2e080efc56" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683" is_pre_election: true
I20260619 12:09:35.988070 2707 raft_consensus.cc:2454] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 0.
I20260619 12:09:35.988070 2987 raft_consensus.cc:2454] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 0.
I20260619 12:09:35.988211 2778 leader_election.cc:302] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:35.988327 3155 raft_consensus.cc:2781] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:35.988399 3155 raft_consensus.cc:482] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:35.988440 3155 raft_consensus.cc:3037] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:35.989099 3155 raft_consensus.cc:504] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.989240 3155 leader_election.cc:288] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 election: Requested vote from peers 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315), b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719)
I20260619 12:09:35.989373 2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6aaca0198b468fbd3b5b2e080efc56" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d5e4c73750483ca02a2014c9c28180"
I20260619 12:09:35.989449 2987 raft_consensus.cc:3037] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:35.989477 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6aaca0198b468fbd3b5b2e080efc56" candidate_uuid: "73aa358785544f6b86db4527961a7adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683"
I20260619 12:09:35.989548 2707 raft_consensus.cc:3037] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:35.990052 2987 raft_consensus.cc:2454] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 1.
I20260619 12:09:35.990219 2707 raft_consensus.cc:2454] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73aa358785544f6b86db4527961a7adc in term 1.
I20260619 12:09:35.990238 2778 leader_election.cc:302] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:35.990345 3155 raft_consensus.cc:2781] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:35.990530 3155 raft_consensus.cc:686] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 1 LEADER]: Becoming Leader. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Running, Role: LEADER
I20260619 12:09:35.990578 3155 consensus_queue.cc:237] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:35.991257 2550 catalog_manager.cc:5495] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc reported cstate change: term changed from 0 to 1, leader changed from <none> to 73aa358785544f6b86db4527961a7adc (127.0.201.2). New cstate: current_term: 1 leader_uuid: "73aa358785544f6b86db4527961a7adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:36.019567 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:36.017333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.0.201.2:42943)
0619 12:09:36.017485 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:36.017487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:36.017495 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:36.017522 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:36.017523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:36.017532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:36.017589 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:36.017684 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.018152 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.018943 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.019130 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.019182 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:36.019377 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:36.019404 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:36.019446 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:36.019492 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 12:09:36.019509 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":110,"threads_started":1}
I20260619 12:09:36.021670 2987 raft_consensus.cc:1261] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Refusing update from remote peer 73aa358785544f6b86db4527961a7adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:36.021663 2707 raft_consensus.cc:1261] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Refusing update from remote peer 73aa358785544f6b86db4527961a7adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:36.021878 3157 consensus_queue.cc:1035] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:36.021981 3155 consensus_queue.cc:1035] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:36.031998 2550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:36.032097 2550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:36.032716 2550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Sending DeleteTablet for 3 replicas of tablet ba6aaca0198b468fbd3b5b2e080efc56
I20260619 12:09:36.032948 2827 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6aaca0198b468fbd3b5b2e080efc56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:36 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:57266
I20260619 12:09:36.032939 2687 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6aaca0198b468fbd3b5b2e080efc56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:36 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49532
I20260619 12:09:36.033056 3139 tablet_replica.cc:330] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: stopping tablet replica
I20260619 12:09:36.033056 3141 tablet_replica.cc:330] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: stopping tablet replica
I20260619 12:09:36.033118 3139 raft_consensus.cc:2229] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:36.033119 3141 raft_consensus.cc:2229] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:36.033167 3141 raft_consensus.cc:2258] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:36.033190 3139 raft_consensus.cc:2258] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:36.033280 2967 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6aaca0198b468fbd3b5b2e080efc56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:36 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36284
I20260619 12:09:36.033427 3140 tablet_replica.cc:330] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: stopping tablet replica
I20260619 12:09:36.033515 3140 raft_consensus.cc:2229] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:36.033560 3140 raft_consensus.cc:2258] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:36.033605 3139 ts_tablet_manager.cc:1905] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:36.033686 2550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:36.033805 2550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:36.033869 3141 ts_tablet_manager.cc:1905] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:36.035431 3139 ts_tablet_manager.cc:1918] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:36.035509 3139 log.cc:1196] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/ba6aaca0198b468fbd3b5b2e080efc56
I20260619 12:09:36.035610 3141 ts_tablet_manager.cc:1918] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:36.035681 3141 log.cc:1196] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/ba6aaca0198b468fbd3b5b2e080efc56
I20260619 12:09:36.035802 3139 ts_tablet_manager.cc:1939] T ba6aaca0198b468fbd3b5b2e080efc56 P 73aa358785544f6b86db4527961a7adc: Deleting consensus metadata
I20260619 12:09:36.035924 3141 ts_tablet_manager.cc:1939] T ba6aaca0198b468fbd3b5b2e080efc56 P 90e95e347c604116a8c3e31cded4f683: Deleting consensus metadata
I20260619 12:09:36.036134 2535 catalog_manager.cc:4841] TS 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943): tablet ba6aaca0198b468fbd3b5b2e080efc56 (table test-table [id=7c49cc0a0c0840ce82003dafc560286b]) successfully deleted
I20260619 12:09:36.036247 2535 catalog_manager.cc:4841] TS 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315): tablet ba6aaca0198b468fbd3b5b2e080efc56 (table test-table [id=7c49cc0a0c0840ce82003dafc560286b]) successfully deleted
I20260619 12:09:36.036545 3140 ts_tablet_manager.cc:1905] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:36.037137 2827 tablet_service.cc:1467] Processing CreateTablet for tablet 63505d3e9f954af6953897d4e67f26dc (DEFAULT_TABLE table=test-table [id=341cbf8bc7df47ea9a0b271ce629b231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:36.037263 2827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63505d3e9f954af6953897d4e67f26dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:36.038305 3140 ts_tablet_manager.cc:1918] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:36.038378 3140 log.cc:1196] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/ba6aaca0198b468fbd3b5b2e080efc56
I20260619 12:09:36.038637 3140 ts_tablet_manager.cc:1939] T ba6aaca0198b468fbd3b5b2e080efc56 P b8d5e4c73750483ca02a2014c9c28180: Deleting consensus metadata
I20260619 12:09:36.038671 3150 tablet_bootstrap.cc:492] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Bootstrap starting.
I20260619 12:09:36.038837 2967 tablet_service.cc:1467] Processing CreateTablet for tablet 63505d3e9f954af6953897d4e67f26dc (DEFAULT_TABLE table=test-table [id=341cbf8bc7df47ea9a0b271ce629b231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:36.038947 3150 tablet_bootstrap.cc:654] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:36.038944 2967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63505d3e9f954af6953897d4e67f26dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:36.038939 2687 tablet_service.cc:1467] Processing CreateTablet for tablet 63505d3e9f954af6953897d4e67f26dc (DEFAULT_TABLE table=test-table [id=341cbf8bc7df47ea9a0b271ce629b231]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:36.039033 2687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63505d3e9f954af6953897d4e67f26dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:36.039462 3150 tablet_bootstrap.cc:492] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: No bootstrap required, opened a new log
I20260619 12:09:36.039525 3150 ts_tablet_manager.cc:1397] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:36.039644 3150 raft_consensus.cc:348] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.039722 3150 raft_consensus.cc:374] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:36.039749 3150 raft_consensus.cc:729] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73aa358785544f6b86db4527961a7adc, State: Initialized, Role: FOLLOWER
I20260619 12:09:36.039789 3150 consensus_queue.cc:260] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.039871 3150 ts_tablet_manager.cc:1428] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:36.040302 3153 tablet_bootstrap.cc:492] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Bootstrap starting.
I20260619 12:09:36.040400 3152 tablet_bootstrap.cc:492] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683: Bootstrap starting.
I20260619 12:09:36.040658 3153 tablet_bootstrap.cc:654] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:36.040710 3152 tablet_bootstrap.cc:654] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:36.041240 2534 catalog_manager.cc:4841] TS b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719): tablet ba6aaca0198b468fbd3b5b2e080efc56 (table test-table [id=7c49cc0a0c0840ce82003dafc560286b]) successfully deleted
I20260619 12:09:36.041599 3152 tablet_bootstrap.cc:492] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683: No bootstrap required, opened a new log
I20260619 12:09:36.041639 3152 ts_tablet_manager.cc:1397] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:36.041754 3153 tablet_bootstrap.cc:492] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: No bootstrap required, opened a new log
I20260619 12:09:36.041764 3152 raft_consensus.cc:348] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.041805 3153 ts_tablet_manager.cc:1397] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:36.041823 3152 raft_consensus.cc:374] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:36.041842 3152 raft_consensus.cc:729] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e95e347c604116a8c3e31cded4f683, State: Initialized, Role: FOLLOWER
I20260619 12:09:36.041888 3152 consensus_queue.cc:260] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.041946 3153 raft_consensus.cc:348] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.042013 3152 ts_tablet_manager.cc:1428] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:36.042019 3153 raft_consensus.cc:374] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:36.042047 3153 raft_consensus.cc:729] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Initialized, Role: FOLLOWER
I20260619 12:09:36.042098 3153 consensus_queue.cc:260] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.042205 3153 ts_tablet_manager.cc:1428] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:36.246958 3126 raft_consensus.cc:482] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:36.247052 3126 raft_consensus.cc:504] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.247267 3126 leader_election.cc:288] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315), 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943)
I20260619 12:09:36.247547 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63505d3e9f954af6953897d4e67f26dc" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683" is_pre_election: true
I20260619 12:09:36.247591 2847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63505d3e9f954af6953897d4e67f26dc" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73aa358785544f6b86db4527961a7adc" is_pre_election: true
I20260619 12:09:36.247669 2707 raft_consensus.cc:2454] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 0.
I20260619 12:09:36.247684 2847 raft_consensus.cc:2454] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 0.
I20260619 12:09:36.247845 2919 leader_election.cc:302] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e95e347c604116a8c3e31cded4f683, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:36.247992 3126 raft_consensus.cc:2781] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:36.248049 3126 raft_consensus.cc:482] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:36.248075 3126 raft_consensus.cc:3037] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:36.248788 3126 raft_consensus.cc:504] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.248929 3126 leader_election.cc:288] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 election: Requested vote from peers 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315), 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943)
I20260619 12:09:36.249095 2707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63505d3e9f954af6953897d4e67f26dc" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e95e347c604116a8c3e31cded4f683"
I20260619 12:09:36.249110 2847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63505d3e9f954af6953897d4e67f26dc" candidate_uuid: "b8d5e4c73750483ca02a2014c9c28180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73aa358785544f6b86db4527961a7adc"
I20260619 12:09:36.249174 2847 raft_consensus.cc:3037] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:36.249176 2707 raft_consensus.cc:3037] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:36.249907 2847 raft_consensus.cc:2454] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 1.
I20260619 12:09:36.249920 2707 raft_consensus.cc:2454] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d5e4c73750483ca02a2014c9c28180 in term 1.
I20260619 12:09:36.250088 2919 leader_election.cc:302] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73aa358785544f6b86db4527961a7adc, b8d5e4c73750483ca02a2014c9c28180; no voters:
I20260619 12:09:36.250195 3126 raft_consensus.cc:2781] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:36.250247 3126 raft_consensus.cc:686] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 1 LEADER]: Becoming Leader. State: Replica: b8d5e4c73750483ca02a2014c9c28180, State: Running, Role: LEADER
I20260619 12:09:36.250294 3126 consensus_queue.cc:237] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } }
I20260619 12:09:36.250886 2549 catalog_manager.cc:5495] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "b8d5e4c73750483ca02a2014c9c28180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d5e4c73750483ca02a2014c9c28180" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45719 } health_report { overall_health: HEALTHY } } }
I20260619 12:09:36.268164 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:36.265425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.0.201.3:45719)
0619 12:09:36.265612 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:36.265614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:36.265622 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:36.265649 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:36.265649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:36.265659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:36.265776 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:36.265894 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.266464 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.267306 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.267603 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.267691 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:36.267861 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:36.267898 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:36.267970 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:36.268036 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 12:09:36.268059 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":139,"threads_started":1}
I20260619 12:09:36.272385 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:36.270134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:52021 (local address 127.0.201.3:45719)
0619 12:09:36.270166 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:36.270169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:36.270208 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:09:36.270275 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:36.270276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:36.270286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:36.270345 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:36.270460 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.270893 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.271725 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.271896 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.271973 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:36.272116 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:36.272148 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:36.272203 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:36.272269 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 12:09:36.272289 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260619 12:09:36.272392 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:36.270033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45719 (local address 127.0.201.3:52021)
0619 12:09:36.270158 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:36.270169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:36.270222 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:36.270360 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:36.270361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:36.270363 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:36.270399 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:36.270400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.270968 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.270969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:36.271603 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:36.271605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:36.271973 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:36.271974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:36.272029 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:36.272033 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:36.272216 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:36.272218 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:36.272265 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:09:36.272306 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260619 12:09:36.274226 2847 raft_consensus.cc:1261] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Refusing update from remote peer b8d5e4c73750483ca02a2014c9c28180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:36.274225 2707 raft_consensus.cc:1261] T 63505d3e9f954af6953897d4e67f26dc P 90e95e347c604116a8c3e31cded4f683 [term 1 FOLLOWER]: Refusing update from remote peer b8d5e4c73750483ca02a2014c9c28180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:36.274426 3064 consensus_queue.cc:1035] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73aa358785544f6b86db4527961a7adc" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:36.274518 3126 consensus_queue.cc:1035] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e95e347c604116a8c3e31cded4f683" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:09:36.278581 3119 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 12:09:37.795751 2549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:37.795843 2549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:37.796603 2549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 322445dd91aa45058adfdf45cd94ad85: Sending DeleteTablet for 3 replicas of tablet 63505d3e9f954af6953897d4e67f26dc
I20260619 12:09:37.796893 2827 tablet_service.cc:1514] Processing DeleteTablet for tablet 63505d3e9f954af6953897d4e67f26dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:37 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:57266
I20260619 12:09:37.796909 2967 tablet_service.cc:1514] Processing DeleteTablet for tablet 63505d3e9f954af6953897d4e67f26dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:37 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36284
I20260619 12:09:37.796952 2687 tablet_service.cc:1514] Processing DeleteTablet for tablet 63505d3e9f954af6953897d4e67f26dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:37 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49532
I20260619 12:09:37.797123 3198 tablet_replica.cc:330] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: stopping tablet replica
I20260619 12:09:37.797201 3198 raft_consensus.cc:2229] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:37.797217 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 2622
I20260619 12:09:37.797353 3196 tablet_replica.cc:330] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: stopping tablet replica
I20260619 12:09:37.797439 3196 raft_consensus.cc:2229] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:37.797578 3196 raft_consensus.cc:2258] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:37.797653 3198 raft_consensus.cc:2258] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:37.797969 3196 ts_tablet_manager.cc:1905] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:37.798215 3198 ts_tablet_manager.cc:1905] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:37.800112 3198 ts_tablet_manager.cc:1918] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:09:37.800105 3196 ts_tablet_manager.cc:1918] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:09:37.800181 3198 log.cc:1196] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/63505d3e9f954af6953897d4e67f26dc
I20260619 12:09:37.800182 3196 log.cc:1196] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/63505d3e9f954af6953897d4e67f26dc
I20260619 12:09:37.800431 3198 ts_tablet_manager.cc:1939] T 63505d3e9f954af6953897d4e67f26dc P 73aa358785544f6b86db4527961a7adc: Deleting consensus metadata
I20260619 12:09:37.800442 3196 ts_tablet_manager.cc:1939] T 63505d3e9f954af6953897d4e67f26dc P b8d5e4c73750483ca02a2014c9c28180: Deleting consensus metadata
I20260619 12:09:37.800735 2534 catalog_manager.cc:4841] TS b8d5e4c73750483ca02a2014c9c28180 (127.0.201.3:45719): tablet 63505d3e9f954af6953897d4e67f26dc (table test-table [id=341cbf8bc7df47ea9a0b271ce629b231]) successfully deleted
I20260619 12:09:37.800791 2535 catalog_manager.cc:4841] TS 73aa358785544f6b86db4527961a7adc (127.0.201.2:42943): tablet 63505d3e9f954af6953897d4e67f26dc (table test-table [id=341cbf8bc7df47ea9a0b271ce629b231]) successfully deleted
W20260619 12:09:37.804076 2535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:38315 (error 108)
I20260619 12:09:37.804311 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 2762
I20260619 12:09:37.804966 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:37.804445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38315 (local address 127.0.0.1:49552)
0619 12:09:37.804895 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:37.804935 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":56,"threads_started":1}
W20260619 12:09:37.805069 2535 catalog_manager.cc:4584] TS 90e95e347c604116a8c3e31cded4f683 (127.0.201.1:38315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 63505d3e9f954af6953897d4e67f26dc: Network error: Client connection negotiation failed: client connection to 127.0.201.1:38315: connect: Connection refused (error 111)
I20260619 12:09:37.810074 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 2902
I20260619 12:09:37.816087 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 2518
2026-06-19T12:09:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6024 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:37 dist-test-slave-c910 krb5kdc[3203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:37 dist-test-slave-c910 krb5kdc[3203](info): set up 2 sockets
Jun 19 12:09:37 dist-test-slave-c910 krb5kdc[3203](info): commencing operation
krb5kdc: starting...
W20260619 12:09:38.846892 804 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:38 dist-test-slave-c910 krb5kdc[3203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:38.958312 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45653
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:45653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:39.037122 3216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:39.037326 3216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:39.037348 3216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:39.038498 3216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:39.038538 3216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:39.038553 3216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:39.038604 3216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:39.038623 3216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:39.040174 3216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:45653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:39.040419 3216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:39.040647 3216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:39.045591 3224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.045629 3222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:39.045679 3216 server_base.cc:1034] running on GCE node
W20260619 12:09:39.045614 3221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:39.045995 3216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:39.046403 3216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:39.047561 3216 hybrid_clock.cc:648] HybridClock initialized: now 1781870979047551 us; error 46 us; skew 500 ppm
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:39.054718 3216 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:39.055143 3216 webserver.cc:460] Webserver started at http://127.0.201.62:36003/ using document root <none> and password file <none>
I20260619 12:09:39.055383 3216 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:39.055454 3216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:39.055605 3216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:39.056694 3216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "f909c2266b544e18917fbee6a0a89a09"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "a39f54f7afa0a56fdad2e5f2b705a2ac"
server_key_iv: "930da6b75e0733bfd7fe8b33aeda1810"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.057051 3216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "f909c2266b544e18917fbee6a0a89a09"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "a39f54f7afa0a56fdad2e5f2b705a2ac"
server_key_iv: "930da6b75e0733bfd7fe8b33aeda1810"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.058346 3216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:39.059095 3231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.059371 3216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:39.059468 3216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "f909c2266b544e18917fbee6a0a89a09"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "a39f54f7afa0a56fdad2e5f2b705a2ac"
server_key_iv: "930da6b75e0733bfd7fe8b33aeda1810"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.059551 3216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:39.073419 3216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:39.075176 3216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:39.075356 3216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:39.078998 3216 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:45653
I20260619 12:09:39.079041 3283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:45653 every 8 connection(s)
I20260619 12:09:39.079603 3216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:39.080233 3284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:39.082404 3284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09: Bootstrap starting.
I20260619 12:09:39.082499 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3216
I20260619 12:09:39.082599 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:39.082777 804 external_mini_cluster.cc:1428] Setting key 89b57edd858a8f45f0f8cfd89d2f8886
I20260619 12:09:39.083052 3284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:39.083420 3284 log.cc:824] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:39.084091 3284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09: No bootstrap required, opened a new log
I20260619 12:09:39.086314 3284 raft_consensus.cc:348] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f909c2266b544e18917fbee6a0a89a09" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45653 } }
I20260619 12:09:39.086472 3284 raft_consensus.cc:374] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:39.086503 3284 raft_consensus.cc:729] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f909c2266b544e18917fbee6a0a89a09, State: Initialized, Role: FOLLOWER
I20260619 12:09:39.086620 3284 consensus_queue.cc:260] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f909c2266b544e18917fbee6a0a89a09" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45653 } }
I20260619 12:09:39.086692 3284 raft_consensus.cc:388] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:39.086719 3284 raft_consensus.cc:482] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:39.086764 3284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:39.087335 3284 raft_consensus.cc:504] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f909c2266b544e18917fbee6a0a89a09" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45653 } }
I20260619 12:09:39.087452 3284 leader_election.cc:302] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f909c2266b544e18917fbee6a0a89a09; no voters:
I20260619 12:09:39.087664 3284 leader_election.cc:288] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:39.087711 3289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:39.087868 3289 raft_consensus.cc:686] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [term 1 LEADER]: Becoming Leader. State: Replica: f909c2266b544e18917fbee6a0a89a09, State: Running, Role: LEADER
I20260619 12:09:39.087939 3284 sys_catalog.cc:564] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:39.088071 3289 consensus_queue.cc:237] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f909c2266b544e18917fbee6a0a89a09" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45653 } }
I20260619 12:09:39.088385 3290 sys_catalog.cc:455] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f909c2266b544e18917fbee6a0a89a09. Latest consensus state: current_term: 1 leader_uuid: "f909c2266b544e18917fbee6a0a89a09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f909c2266b544e18917fbee6a0a89a09" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45653 } } }
I20260619 12:09:39.088490 3290 sys_catalog.cc:458] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:39.089030 3294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:39.089499 3294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:39.090188 3291 sys_catalog.cc:455] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f909c2266b544e18917fbee6a0a89a09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f909c2266b544e18917fbee6a0a89a09" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45653 } } }
I20260619 12:09:39.090325 3291 sys_catalog.cc:458] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:39.091056 3294 catalog_manager.cc:1347] Generated new cluster ID: 71b9530ce172440ba2272757393e4f14
I20260619 12:09:39.091109 3294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:39.093550 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.083351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.0.201.62:45653)
0619 12:09:39.083491 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:39.083500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:39.083532 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:09:39.084047 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:39.084055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:39.084103 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:39.084249 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:39.084989 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.085817 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.086899 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.087338 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.087439 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.090446 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:39.090449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:39.090450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:39.090461 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:39.092286 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.092628 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.092630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.092631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.092749 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.092933 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.092933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.092933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.093130 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:39.093223 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:39.093331 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 12:09:39.093421 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
I20260619 12:09:39.098480 3294 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:39.099298 3294 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:39.103004 3294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f909c2266b544e18917fbee6a0a89a09: Generated new TSK 0
I20260619 12:09:39.103154 3294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:39.130110 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45653
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:39.209035 3310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:39.209254 3310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:39.209276 3310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:39.210458 3310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:39.210500 3310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:39.210526 3310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:39.212020 3310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:39.212255 3310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:39.212610 3310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:39.218309 3316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.218318 3318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.218318 3315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:39.218603 3310 server_base.cc:1034] running on GCE node
I20260619 12:09:39.218830 3310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:39.219062 3310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:39.220217 3310 hybrid_clock.cc:648] HybridClock initialized: now 1781870979220214 us; error 48 us; skew 500 ppm
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:39.222620 3310 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:39.223038 3310 webserver.cc:460] Webserver started at http://127.0.201.1:35265/ using document root <none> and password file <none>
I20260619 12:09:39.223332 3310 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:39.223402 3310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:39.223536 3310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:39.224650 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "1c33a0c6bc9143fe9ee0f5bbe9e10a8a"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "024530cc0e52445d6070f75d0b065af5"
server_key_iv: "a40457e2e7615154e935a8bde2f79884"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.225009 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "1c33a0c6bc9143fe9ee0f5bbe9e10a8a"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "024530cc0e52445d6070f75d0b065af5"
server_key_iv: "a40457e2e7615154e935a8bde2f79884"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.226341 3310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:39.227098 3325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.227365 3310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:39.227464 3310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "1c33a0c6bc9143fe9ee0f5bbe9e10a8a"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "024530cc0e52445d6070f75d0b065af5"
server_key_iv: "a40457e2e7615154e935a8bde2f79884"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.227550 3310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:39.235271 3310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:39.238301 3310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:39.238507 3310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:39.238749 3310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:39.239101 3310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:39.239148 3310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.239212 3310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:39.239241 3310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.245448 3310 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:35889
I20260619 12:09:39.245487 3438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:35889 every 8 connection(s)
I20260619 12:09:39.245810 3310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:39.254318 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.246530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47927 (local address 127.0.201.62:45653)
0619 12:09:39.246606 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:39.246609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:39.246779 (+ 170us) server_negotiation.cc:407] Connection header received
0619 12:09:39.247733 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:39.247734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:39.247745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:39.247799 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:39.248385 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.248891 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.249828 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.250026 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.250081 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.252077 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:39.252078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:39.252078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:39.252085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:39.253231 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.253577 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.253578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.253578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.253623 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.253821 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.253821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.253822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.253920 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:39.253969 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:39.254183 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 12:09:39.254244 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:39.254557 3441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.246495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45653 (local address 127.0.201.1:47927)
0619 12:09:39.246660 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:39.246695 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:39.247639 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:39.247872 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:39.247875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:39.248116 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:39.248309 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:39.248313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.248966 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.248967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:39.249750 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:39.249751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.250098 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.250098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:39.250190 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.250630 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:39.250637 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:39.251998 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:39.253299 (+ 1301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:39.253304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:39.253306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:39.253516 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:39.253678 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:39.253679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:39.253679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:39.253769 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:39.253972 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:39.253976 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:39.254081 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 12:09:39.254159 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:09:39.254305 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260619 12:09:39.254547 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3310
I20260619 12:09:39.254633 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:39.254832 804 external_mini_cluster.cc:1428] Setting key 286f1ae624786e774a5add77212c70df
I20260619 12:09:39.255059 3439 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45653
I20260619 12:09:39.255160 3439 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:39.255371 3439 heartbeater.cc:511] Master 127.0.201.62:45653 requested a full tablet report, sending...
I20260619 12:09:39.255955 3248 ts_manager.cc:194] Registered new tserver with Master: 1c33a0c6bc9143fe9ee0f5bbe9e10a8a (127.0.201.1:35889)
I20260619 12:09:39.256858 3248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:47927
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:39.278143 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45653
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:39.370522 3444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:39.370877 3444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:39.370919 3444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:39.372949 3444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:39.372995 3444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:39.373025 3444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:39.374667 3444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:39.374974 3444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:39.375310 3444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:39.380609 3450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.380620 3449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.380609 3452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:39.381294 3444 server_base.cc:1034] running on GCE node
I20260619 12:09:39.381526 3444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:39.381774 3444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:39.382953 3444 hybrid_clock.cc:648] HybridClock initialized: now 1781870979382936 us; error 36 us; skew 500 ppm
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:39.385670 3444 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:39.386195 3444 webserver.cc:460] Webserver started at http://127.0.201.2:43099/ using document root <none> and password file <none>
I20260619 12:09:39.386463 3444 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:39.386514 3444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:39.386665 3444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:39.387881 3444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "d95da605769a4a17bc87bd87848c8ea9"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "05a963e7430f7cf641462c27298fc74c"
server_key_iv: "9ebbdf231cfff0aa7f28f99a01c247d2"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.388365 3444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "d95da605769a4a17bc87bd87848c8ea9"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "05a963e7430f7cf641462c27298fc74c"
server_key_iv: "9ebbdf231cfff0aa7f28f99a01c247d2"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.389959 3444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:39.390816 3459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.390969 3444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:39.391068 3444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "d95da605769a4a17bc87bd87848c8ea9"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "05a963e7430f7cf641462c27298fc74c"
server_key_iv: "9ebbdf231cfff0aa7f28f99a01c247d2"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.391140 3444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:39.405220 3444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:39.407445 3444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:39.407595 3444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:39.407837 3444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:39.408147 3444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:39.408181 3444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.408202 3444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:39.408241 3444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.415377 3444 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:41307
I20260619 12:09:39.415501 3572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:41307 every 8 connection(s)
I20260619 12:09:39.415792 3444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:39.423164 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3444
I20260619 12:09:39.423295 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:39.423520 804 external_mini_cluster.cc:1428] Setting key 2f8349cd692556dc6b6c060d03a5ed66
I20260619 12:09:39.426578 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.416552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:58005 (local address 127.0.201.62:45653)
0619 12:09:39.416618 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:39.416621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:39.416780 (+ 159us) server_negotiation.cc:407] Connection header received
0619 12:09:39.417685 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:39.417686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:39.417699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:39.417789 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:39.418371 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.419327 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.420327 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.420696 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.420772 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.422828 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:39.422830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:39.422830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:39.422838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:39.424854 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.425350 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.425351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.425352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.425433 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.425775 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.425777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.425777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.425930 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:39.426008 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:39.426365 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 12:09:39.426458 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:09:39.426792 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.416508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45653 (local address 127.0.201.2:58005)
0619 12:09:39.416648 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:39.416696 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:39.417585 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:39.417842 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:39.417845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:39.418084 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:39.418289 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:39.418293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.419419 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.419420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:39.420239 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:39.420241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.420780 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.420780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:39.420877 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.421400 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:39.421416 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:39.422740 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:39.424938 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:39.424945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:39.424948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:39.425270 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:39.425519 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:39.425520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:39.425520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:39.425666 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:39.426011 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:39.426017 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:39.426197 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 12:09:39.426332 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 12:09:39.426514 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260619 12:09:39.427222 3573 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45653
I20260619 12:09:39.427318 3573 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:39.427487 3573 heartbeater.cc:511] Master 127.0.201.62:45653 requested a full tablet report, sending...
I20260619 12:09:39.427882 3248 ts_manager.cc:194] Registered new tserver with Master: d95da605769a4a17bc87bd87848c8ea9 (127.0.201.2:41307)
I20260619 12:09:39.428264 3248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:58005
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:39.453511 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45653
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:39.531682 3578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:39.531898 3578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:39.531932 3578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:39.533138 3578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:39.533196 3578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:39.533248 3578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:39.534703 3578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:39.534960 3578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:39.535174 3578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:39.539819 3583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.539866 3586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:39.539901 3584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:39.540112 3578 server_base.cc:1034] running on GCE node
I20260619 12:09:39.540339 3578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:39.540531 3578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:39.541684 3578 hybrid_clock.cc:648] HybridClock initialized: now 1781870979541670 us; error 28 us; skew 500 ppm
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:39.544466 3578 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:39.544906 3578 webserver.cc:460] Webserver started at http://127.0.201.3:36493/ using document root <none> and password file <none>
I20260619 12:09:39.545077 3578 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:39.545125 3578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:39.545209 3578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:39.546190 3578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "d28245924996411c8bec9c19adf56d05"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "638ed2699f879beee7e3fc9b730681af"
server_key_iv: "7811a3b5f5d4a4834a210e734a3336c9"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.546479 3578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "d28245924996411c8bec9c19adf56d05"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "638ed2699f879beee7e3fc9b730681af"
server_key_iv: "7811a3b5f5d4a4834a210e734a3336c9"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.547783 3578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:39.548496 3593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.548671 3578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:39.548774 3578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "d28245924996411c8bec9c19adf56d05"
format_stamp: "Formatted at 2026-06-19 12:09:39 on dist-test-slave-c910"
server_key: "638ed2699f879beee7e3fc9b730681af"
server_key_iv: "7811a3b5f5d4a4834a210e734a3336c9"
server_key_version: "encryptionkey@0"
I20260619 12:09:39.548843 3578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:39.563877 3578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:39.566059 3578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:39.566213 3578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:39.566416 3578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:39.566715 3578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:39.566766 3578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.566804 3578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:39.566831 3578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:39.572523 3578 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:33927
I20260619 12:09:39.572559 3706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:33927 every 8 connection(s)
I20260619 12:09:39.572876 3578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:39.577771 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3578
I20260619 12:09:39.577877 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:39.578089 804 external_mini_cluster.cc:1428] Setting key 49a4f843b5adb1c4cdc9d6b1592cab85
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:39.582258 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.573634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:45961 (local address 127.0.201.62:45653)
0619 12:09:39.573693 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:39.573695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:39.573848 (+ 153us) server_negotiation.cc:407] Connection header received
0619 12:09:39.574726 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:39.574726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:39.574735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:39.574802 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:39.575461 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.575949 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.576998 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.577185 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.577236 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.579596 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:39.579597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:39.579597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:39.579604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:39.581009 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.581391 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.581392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.581392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.581448 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:39.581705 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:39.581706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:39.581706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:39.581827 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:39.581899 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:39.582131 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 12:09:39.582196 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:09:39.582360 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:39.573584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45653 (local address 127.0.201.3:45961)
0619 12:09:39.573727 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:39.573766 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:39.574635 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:39.574870 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:39.574873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:39.575110 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:39.575389 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:39.575393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.576020 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.576021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:39.576917 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:39.576925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:39.577257 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:39.577258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:39.577356 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:39.577910 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:39.577927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:39.579512 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:39.581074 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:39.581078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:39.581079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:39.581285 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:39.581522 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:39.581522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:39.581523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:39.581608 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:39.581894 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:39.581898 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:39.581999 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 12:09:39.582072 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:09:39.582178 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260619 12:09:39.582861 3707 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45653
I20260619 12:09:39.583035 3707 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:39.583285 3707 heartbeater.cc:511] Master 127.0.201.62:45653 requested a full tablet report, sending...
I20260619 12:09:39.583732 3248 ts_manager.cc:194] Registered new tserver with Master: d28245924996411c8bec9c19adf56d05 (127.0.201.3:33927)
I20260619 12:09:39.584113 3248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:45961
I20260619 12:09:39.589718 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:09:39.600544 3287 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:39.599617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.0.201.62:45653)
0619 12:09:39.599677 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:39.599680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:39.599817 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:09:39.599873 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:39.599873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:39.599882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:39.599937 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:39.600451 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34482: BlockingRecv error: recv got EOF from 127.0.0.1:34482 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:09:39.601425 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3310
I20260619 12:09:39.605769 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3444
I20260619 12:09:39.609879 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3578
I20260619 12:09:39.614217 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3216
2026-06-19T12:09:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1797 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3719](info): set up 2 sockets
Jun 19 12:09:39 dist-test-slave-c910 krb5kdc[3719](info): commencing operation
krb5kdc: starting...
W20260619 12:09:40.642956 804 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:40 dist-test-slave-c910 krb5kdc[3719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:40.751596 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:40305
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:40305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:40.828125 3732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:40.828354 3732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:40.828389 3732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:40.829512 3732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:40.829579 3732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:40.829603 3732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:40.829627 3732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:40.829656 3732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:40.831066 3732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:40305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:40305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:40.831360 3732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:40.831575 3732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:40.836241 3738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:40.836517 3732 server_base.cc:1034] running on GCE node
W20260619 12:09:40.836197 3737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:40.836192 3740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:40.836858 3732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:40.837137 3732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:40.838286 3732 hybrid_clock.cc:648] HybridClock initialized: now 1781870980838278 us; error 31 us; skew 500 ppm
Jun 19 12:09:40 dist-test-slave-c910 krb5kdc[3719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870980, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:40.840677 3732 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:40.841102 3732 webserver.cc:460] Webserver started at http://127.0.201.62:40607/ using document root <none> and password file <none>
I20260619 12:09:40.841332 3732 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:40.841398 3732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:40.841547 3732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:40.842558 3732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "92751e7af7804c29bc3e5c94e8d59d62"
format_stamp: "Formatted at 2026-06-19 12:09:40 on dist-test-slave-c910"
server_key: "629447bc9f2243227d5fa06e02c627d4"
server_key_iv: "9c8b7de9fd9859ba601d384e68a3f223"
server_key_version: "encryptionkey@0"
I20260619 12:09:40.842911 3732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "92751e7af7804c29bc3e5c94e8d59d62"
format_stamp: "Formatted at 2026-06-19 12:09:40 on dist-test-slave-c910"
server_key: "629447bc9f2243227d5fa06e02c627d4"
server_key_iv: "9c8b7de9fd9859ba601d384e68a3f223"
server_key_version: "encryptionkey@0"
I20260619 12:09:40.844295 3732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 12:09:40.845077 3747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:40.845288 3732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:40.845382 3732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "92751e7af7804c29bc3e5c94e8d59d62"
format_stamp: "Formatted at 2026-06-19 12:09:40 on dist-test-slave-c910"
server_key: "629447bc9f2243227d5fa06e02c627d4"
server_key_iv: "9c8b7de9fd9859ba601d384e68a3f223"
server_key_version: "encryptionkey@0"
I20260619 12:09:40.845466 3732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:40.865567 3732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:40.867646 3732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:40.867810 3732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:40.871632 3732 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:40305
I20260619 12:09:40.871673 3799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:40305 every 8 connection(s)
I20260619 12:09:40.872022 3732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:40.872829 3800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:40.875095 3800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62: Bootstrap starting.
I20260619 12:09:40.875635 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3732
I20260619 12:09:40.875735 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:40.875795 3800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:40.875933 804 external_mini_cluster.cc:1428] Setting key 48be6d96b508690857758a4428ec0dfe
I20260619 12:09:40.876135 3800 log.cc:824] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:40.876883 3800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62: No bootstrap required, opened a new log
I20260619 12:09:40.879136 3800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92751e7af7804c29bc3e5c94e8d59d62" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 40305 } }
I20260619 12:09:40.879339 3800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:40.879375 3800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92751e7af7804c29bc3e5c94e8d59d62, State: Initialized, Role: FOLLOWER
I20260619 12:09:40.879494 3800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92751e7af7804c29bc3e5c94e8d59d62" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 40305 } }
I20260619 12:09:40.879594 3800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:40.879638 3800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:40.879684 3800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:40.880216 3800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92751e7af7804c29bc3e5c94e8d59d62" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 40305 } }
I20260619 12:09:40.880319 3800 leader_election.cc:302] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92751e7af7804c29bc3e5c94e8d59d62; no voters:
I20260619 12:09:40.880455 3800 leader_election.cc:288] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:40.880563 3805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:40.880725 3800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:40.880710 3805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [term 1 LEADER]: Becoming Leader. State: Replica: 92751e7af7804c29bc3e5c94e8d59d62, State: Running, Role: LEADER
I20260619 12:09:40.880879 3805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92751e7af7804c29bc3e5c94e8d59d62" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 40305 } }
I20260619 12:09:40.881249 3806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92751e7af7804c29bc3e5c94e8d59d62. Latest consensus state: current_term: 1 leader_uuid: "92751e7af7804c29bc3e5c94e8d59d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92751e7af7804c29bc3e5c94e8d59d62" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 40305 } } }
I20260619 12:09:40.881340 3806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:40.881543 3813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:40.882028 3813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:40.883142 3807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92751e7af7804c29bc3e5c94e8d59d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92751e7af7804c29bc3e5c94e8d59d62" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 40305 } } }
I20260619 12:09:40.883235 3807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62 [sys.catalog]: This master's current role is: LEADER
Jun 19 12:09:40 dist-test-slave-c910 krb5kdc[3719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:40.883694 3813 catalog_manager.cc:1347] Generated new cluster ID: 0c3e0080b2f141bd9d484574a0b447c4
I20260619 12:09:40.883741 3813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:40.887101 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:40.876395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36094 (local address 127.0.201.62:40305)
0619 12:09:40.876561 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:40.876575 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:40.876607 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:09:40.877171 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:40.877180 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:40.877242 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:40.877395 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:40.877968 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:40.879380 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:40.880239 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:40.880848 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:40.881003 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:40.884056 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:40.884060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:40.884060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:40.884071 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:40.885669 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:40.886032 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:40.886035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:40.886035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:40.886117 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:40.886319 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:40.886320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:40.886320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:40.886537 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:40.886708 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:40.886793 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 12:09:40.886924 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260619 12:09:40.887996 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:40.894029 3813 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:40.894683 3813 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:40.900876 3813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92751e7af7804c29bc3e5c94e8d59d62: Generated new TSK 0
I20260619 12:09:40.901055 3813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:40.901348 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:40.897851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.0.201.62:40305)
0619 12:09:40.897966 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:40.897968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:40.898024 (+ 56us) server_negotiation.cc:407] Connection header received
0619 12:09:40.898098 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:40.898099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:40.898109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:40.898164 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:40.898498 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:40.898985 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:40.899789 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:40.899988 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:40.900070 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:40.900812 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:40.900814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:40.900814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:40.901019 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:40.901036 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:40.901043 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:40.901116 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:40.901246 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 12:09:40.901273 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260619 12:09:40.903693 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3732
2026-06-19T12:09:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1286 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:40 dist-test-slave-c910 krb5kdc[3833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:40 dist-test-slave-c910 krb5kdc[3833](info): set up 2 sockets
Jun 19 12:09:40 dist-test-slave-c910 krb5kdc[3833](info): commencing operation
krb5kdc: starting...
W20260619 12:09:41.931129 804 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:42.042393 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:39749
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:39749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:42.121390 3846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:42.121580 3846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:42.121603 3846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:42.122718 3846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:42.122759 3846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:42.122773 3846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:42.122786 3846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:42.122843 3846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:42.124264 3846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:39749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:42.124526 3846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:42.124749 3846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:42.129454 3852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.129520 3846 server_base.cc:1034] running on GCE node
W20260619 12:09:42.129467 3854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:42.129489 3851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.129930 3846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:42.130224 3846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:42.131394 3846 hybrid_clock.cc:648] HybridClock initialized: now 1781870982131391 us; error 49 us; skew 500 ppm
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:42.133771 3846 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:42.134260 3846 webserver.cc:460] Webserver started at http://127.0.201.62:36629/ using document root <none> and password file <none>
I20260619 12:09:42.134490 3846 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:42.134569 3846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:42.134706 3846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:42.135838 3846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "495bd6b9b79940f28291495288b679fa"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "0eeaeb163774c8a2a40718c784cf68f8"
server_key_iv: "b854906be47c470dde83baac0fc9d92c"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.136219 3846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "495bd6b9b79940f28291495288b679fa"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "0eeaeb163774c8a2a40718c784cf68f8"
server_key_iv: "b854906be47c470dde83baac0fc9d92c"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.137528 3846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:42.138257 3861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.138444 3846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:42.138563 3846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "495bd6b9b79940f28291495288b679fa"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "0eeaeb163774c8a2a40718c784cf68f8"
server_key_iv: "b854906be47c470dde83baac0fc9d92c"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.138638 3846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:42.152308 3846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:42.154032 3846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:42.154134 3846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:42.158134 3846 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:39749
I20260619 12:09:42.158162 3913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:39749 every 8 connection(s)
I20260619 12:09:42.158491 3846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:42.159466 3914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:42.161540 3914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa: Bootstrap starting.
I20260619 12:09:42.162027 3914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:42.162256 3914 log.cc:824] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:42.162858 3914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa: No bootstrap required, opened a new log
I20260619 12:09:42.164394 3914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495bd6b9b79940f28291495288b679fa" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39749 } }
I20260619 12:09:42.164521 3914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:42.164542 3914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495bd6b9b79940f28291495288b679fa, State: Initialized, Role: FOLLOWER
I20260619 12:09:42.164669 3914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495bd6b9b79940f28291495288b679fa" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39749 } }
I20260619 12:09:42.164748 3914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:42.164770 3914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:42.164798 3914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:42.165500 3914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495bd6b9b79940f28291495288b679fa" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39749 } }
I20260619 12:09:42.165582 3914 leader_election.cc:302] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 495bd6b9b79940f28291495288b679fa; no voters:
I20260619 12:09:42.165691 3914 leader_election.cc:288] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:42.165757 3917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:42.165925 3917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [term 1 LEADER]: Becoming Leader. State: Replica: 495bd6b9b79940f28291495288b679fa, State: Running, Role: LEADER
I20260619 12:09:42.165969 3914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:42.166009 3917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495bd6b9b79940f28291495288b679fa" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39749 } }
I20260619 12:09:42.166405 3918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "495bd6b9b79940f28291495288b679fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495bd6b9b79940f28291495288b679fa" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39749 } } }
I20260619 12:09:42.166829 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3846
I20260619 12:09:42.166929 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:42.167133 804 external_mini_cluster.cc:1428] Setting key 24c0c13c1d5ee2888e2d32edaee542d2
I20260619 12:09:42.166441 3919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 495bd6b9b79940f28291495288b679fa. Latest consensus state: current_term: 1 leader_uuid: "495bd6b9b79940f28291495288b679fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495bd6b9b79940f28291495288b679fa" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39749 } } }
I20260619 12:09:42.167316 3919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:42.167497 3918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:42.167512 3932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:42.168362 3932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:42.169775 3932 catalog_manager.cc:1347] Generated new cluster ID: aef124e248424eca942a15556f74d23d
I20260619 12:09:42.169826 3932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:42.178294 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.167605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.0.201.62:39749)
0619 12:09:42.167798 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:42.167807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:42.167843 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:09:42.167958 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:42.167965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:42.168018 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:42.168171 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:42.168906 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.170182 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.171285 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.171871 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.171963 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.174623 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:42.174627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:42.174628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:42.174639 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:42.176683 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.177058 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.177060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.177061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.177180 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.177375 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.177376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.177377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.177703 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:42.177853 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:42.177950 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 12:09:42.178082 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":55,"threads_started":1}
I20260619 12:09:42.184018 3932 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:42.184684 3932 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:42.188396 3932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 495bd6b9b79940f28291495288b679fa: Generated new TSK 0
I20260619 12:09:42.188550 3932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:42.214370 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39749
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:42.294478 3940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:42.294675 3940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:42.294697 3940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:42.295854 3940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:42.295897 3940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:42.295922 3940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:42.297293 3940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:42.297518 3940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:42.297746 3940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:42.302654 3946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:42.302675 3948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.302841 3940 server_base.cc:1034] running on GCE node
W20260619 12:09:42.302675 3945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.303117 3940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:42.303365 3940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:42.304524 3940 hybrid_clock.cc:648] HybridClock initialized: now 1781870982304519 us; error 42 us; skew 500 ppm
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:42.306945 3940 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:42.307426 3940 webserver.cc:460] Webserver started at http://127.0.201.1:33367/ using document root <none> and password file <none>
I20260619 12:09:42.307653 3940 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:42.307719 3940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:42.307873 3940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:42.308956 3940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "099753653df541e58d80a9f37fe82ca5"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "db41be0832d3fda7de9cbb7aca618ac2"
server_key_iv: "e1664a1c15e965352d6c27aaee7ede19"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.309299 3940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "099753653df541e58d80a9f37fe82ca5"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "db41be0832d3fda7de9cbb7aca618ac2"
server_key_iv: "e1664a1c15e965352d6c27aaee7ede19"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.310652 3940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:42.311604 3955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.311806 3940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:42.311908 3940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "099753653df541e58d80a9f37fe82ca5"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "db41be0832d3fda7de9cbb7aca618ac2"
server_key_iv: "e1664a1c15e965352d6c27aaee7ede19"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.311987 3940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:42.340229 3940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:42.342810 3940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:42.342983 3940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:42.343278 3940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:42.343602 3940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:42.343659 3940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.343701 3940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:42.343727 3940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.349345 3940 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:38453
I20260619 12:09:42.349406 4068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:38453 every 8 connection(s)
I20260619 12:09:42.349741 3940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:42.359155 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 3940
I20260619 12:09:42.359304 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:42.359396 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.350497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:37127 (local address 127.0.201.62:39749)
0619 12:09:42.350558 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:42.350561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:42.350783 (+ 222us) server_negotiation.cc:407] Connection header received
0619 12:09:42.351421 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:42.351423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:42.351440 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:42.351512 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:42.352457 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.352969 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.353998 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.354215 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.354325 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.356577 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:42.356578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:42.356579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:42.356587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:42.357927 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.358408 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.358409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.358410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.358469 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.358740 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.358741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.358741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.358857 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:42.358930 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:42.359221 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 12:09:42.359305 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:09:42.359568 804 external_mini_cluster.cc:1428] Setting key f16b942218f9d78df4b69150e04ba0e8
I20260619 12:09:42.359597 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.350472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:39749 (local address 127.0.201.1:37127)
0619 12:09:42.350647 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:42.350692 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:42.351286 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:42.351569 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:42.351573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:42.352151 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:42.352342 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:42.352347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.353086 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.353088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:42.353895 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:42.353897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.354344 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.354345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:42.354474 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.354932 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:42.354940 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:42.356461 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:42.358080 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:42.358086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:42.358088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:42.358313 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:42.358539 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:42.358540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:42.358540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:42.358653 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:42.358934 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:42.358941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:42.359076 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 12:09:42.359183 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:09:42.359359 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260619 12:09:42.360304 4069 heartbeater.cc:347] Connected to a master server at 127.0.201.62:39749
I20260619 12:09:42.360383 4069 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:42.360565 4069 heartbeater.cc:511] Master 127.0.201.62:39749 requested a full tablet report, sending...
I20260619 12:09:42.361107 3878 ts_manager.cc:194] Registered new tserver with Master: 099753653df541e58d80a9f37fe82ca5 (127.0.201.1:38453)
I20260619 12:09:42.361704 3878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:37127
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:42.385648 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39749
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:42.467099 4074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:42.467370 4074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:42.467394 4074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:42.468580 4074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:42.468624 4074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:42.468652 4074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:42.470001 4074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:42.470175 4074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:42.470342 4074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:42.475515 4082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:42.475572 4079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:42.475533 4080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.475961 4074 server_base.cc:1034] running on GCE node
I20260619 12:09:42.476192 4074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:42.476425 4074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:42.477596 4074 hybrid_clock.cc:648] HybridClock initialized: now 1781870982477592 us; error 40 us; skew 500 ppm
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:42.480154 4074 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:42.480602 4074 webserver.cc:460] Webserver started at http://127.0.201.2:33959/ using document root <none> and password file <none>
I20260619 12:09:42.480832 4074 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:42.480901 4074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:42.481050 4074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:42.482064 4074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "d3ee84dcb99e460cb2463374033af8e1"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "5fdd169f56ec4489226029cee4329144"
server_key_iv: "ba83ae0052ea93523c2a22b7bae19181"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.482417 4074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "d3ee84dcb99e460cb2463374033af8e1"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "5fdd169f56ec4489226029cee4329144"
server_key_iv: "ba83ae0052ea93523c2a22b7bae19181"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.483776 4074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 12:09:42.484529 4089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.484694 4074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:42.484787 4074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "d3ee84dcb99e460cb2463374033af8e1"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "5fdd169f56ec4489226029cee4329144"
server_key_iv: "ba83ae0052ea93523c2a22b7bae19181"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.484866 4074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:42.492273 4074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:42.493959 4074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:42.494122 4074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:42.494341 4074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:42.494632 4074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:42.494686 4074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.494726 4074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:42.494767 4074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.500490 4074 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:40619
I20260619 12:09:42.500535 4202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:40619 every 8 connection(s)
I20260619 12:09:42.500840 4074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:42.509428 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.501467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:52219 (local address 127.0.201.62:39749)
0619 12:09:42.501530 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:42.501532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:42.501742 (+ 210us) server_negotiation.cc:407] Connection header received
0619 12:09:42.502229 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:42.502230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:42.502239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:42.502288 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:42.503137 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.503660 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.504618 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.504794 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.504842 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.506904 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:42.506905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:42.506906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:42.506912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:42.508345 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.508698 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.508699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.508699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.508748 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.508944 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.508944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.508944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.509039 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:42.509086 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:42.509305 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 12:09:42.509366 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:09:42.509598 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.501418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:39749 (local address 127.0.201.2:52219)
0619 12:09:42.501603 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:42.501652 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:42.502135 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:42.502338 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:42.502341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:42.502871 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:42.503049 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:42.503066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.503739 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.503740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:42.504538 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:42.504540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.504862 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.504863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:42.504957 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.505429 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:42.505437 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:42.506815 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:42.508411 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:42.508416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:42.508417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:42.508634 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:42.508802 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:42.508802 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:42.508803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:42.508892 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:42.509092 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:42.509095 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:42.509200 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 12:09:42.509280 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 12:09:42.509391 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":62,"threads_started":1}
I20260619 12:09:42.510066 4203 heartbeater.cc:347] Connected to a master server at 127.0.201.62:39749
I20260619 12:09:42.510179 4203 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:42.510354 4203 heartbeater.cc:511] Master 127.0.201.62:39749 requested a full tablet report, sending...
I20260619 12:09:42.510414 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4074
I20260619 12:09:42.510524 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:42.510764 804 external_mini_cluster.cc:1428] Setting key 75f73cb57cc66ea3084a03e4ce18bb6e
I20260619 12:09:42.510762 3878 ts_manager.cc:194] Registered new tserver with Master: d3ee84dcb99e460cb2463374033af8e1 (127.0.201.2:40619)
I20260619 12:09:42.511256 3878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:52219
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:42.533857 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39749
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:42.612970 4208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:42.613168 4208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:42.613190 4208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:42.614248 4208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:42.614280 4208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:42.614306 4208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:42.615736 4208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:42.615981 4208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:42.616207 4208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:42.620996 4214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:42.621009 4213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.621182 4208 server_base.cc:1034] running on GCE node
W20260619 12:09:42.621068 4216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:42.621434 4208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:42.621645 4208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:42.622786 4208 hybrid_clock.cc:648] HybridClock initialized: now 1781870982622765 us; error 38 us; skew 500 ppm
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:42.625217 4208 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:42.625573 4208 webserver.cc:460] Webserver started at http://127.0.201.3:45061/ using document root <none> and password file <none>
I20260619 12:09:42.625759 4208 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:42.625799 4208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:42.625886 4208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:42.626847 4208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "d1394038fcd647dfbde253e57ab9293f"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "0445cdbfb642e03433991e6fd58f674d"
server_key_iv: "1b0ee0fe123e8e909fb2e5dd9a8a2dc3"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.627122 4208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "d1394038fcd647dfbde253e57ab9293f"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "0445cdbfb642e03433991e6fd58f674d"
server_key_iv: "1b0ee0fe123e8e909fb2e5dd9a8a2dc3"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.628486 4208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:42.629153 4223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.629376 4208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:42.629462 4208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "d1394038fcd647dfbde253e57ab9293f"
format_stamp: "Formatted at 2026-06-19 12:09:42 on dist-test-slave-c910"
server_key: "0445cdbfb642e03433991e6fd58f674d"
server_key_iv: "1b0ee0fe123e8e909fb2e5dd9a8a2dc3"
server_key_version: "encryptionkey@0"
I20260619 12:09:42.629529 4208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:42.642755 4208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:42.645013 4208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:42.645167 4208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:42.645390 4208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:42.645680 4208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:42.645735 4208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.645776 4208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:42.645804 4208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:42.651479 4208 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:43295
I20260619 12:09:42.651531 4336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:43295 every 8 connection(s)
I20260619 12:09:42.651862 4208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:42.658397 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4208
I20260619 12:09:42.658500 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:42.658705 804 external_mini_cluster.cc:1428] Setting key 2e6fe7959c68ca1e19b33445ffa54d67
I20260619 12:09:42.661367 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.652681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:55929 (local address 127.0.201.62:39749)
0619 12:09:42.652892 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:42.652894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:42.653091 (+ 197us) server_negotiation.cc:407] Connection header received
0619 12:09:42.653593 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:42.653594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:42.653606 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:42.653653 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:42.654613 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.655146 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.656174 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.656380 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.656465 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.658517 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:42.658518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:42.658519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:42.658527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:42.659993 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.660427 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.660429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.660429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.660489 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.660729 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.660730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.660730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.660845 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:42.660917 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:42.661206 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:09:42.661282 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260619 12:09:42.661554 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.652607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:39749 (local address 127.0.201.3:55929)
0619 12:09:42.652925 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:42.652968 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:42.653467 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:42.653739 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:42.653742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:42.654293 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:42.654479 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:42.654483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.655273 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.655275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:42.656081 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:42.656083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.656473 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.656474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:42.656609 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.657042 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:42.657051 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:42.658421 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:42.660116 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:42.660122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:42.660124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:42.660337 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:42.660550 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:42.660551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:42.660551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:42.660663 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:42.660908 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:42.660913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:42.661055 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 12:09:42.661162 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:09:42.661315 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":101,"threads_started":1}
I20260619 12:09:42.662022 4337 heartbeater.cc:347] Connected to a master server at 127.0.201.62:39749
I20260619 12:09:42.662101 4337 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:42.662282 4337 heartbeater.cc:511] Master 127.0.201.62:39749 requested a full tablet report, sending...
I20260619 12:09:42.662626 3877 ts_manager.cc:194] Registered new tserver with Master: d1394038fcd647dfbde253e57ab9293f (127.0.201.3:43295)
I20260619 12:09:42.663012 3877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:55929
I20260619 12:09:42.670303 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870982, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:42.692246 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:42.685123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.0.201.62:39749)
0619 12:09:42.685208 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:42.685210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:42.685283 (+ 73us) server_negotiation.cc:407] Connection header received
0619 12:09:42.685375 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:42.685376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:42.685386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:42.685465 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:42.685983 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.686470 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.687301 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:42.687521 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:42.687610 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:42.689901 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:42.689902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:42.689903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:42.689910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:42.691068 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.691428 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.691429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.691430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.691492 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:42.691690 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:42.691691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:42.691691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:42.691839 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:42.691944 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:42.692050 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 12:09:42.692129 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 12:09:42.692514 3877 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54168
I20260619 12:09:42.693013 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3940
I20260619 12:09:42.697535 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4074
I20260619 12:09:42.701566 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4208
I20260619 12:09:42.705667 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 3846
2026-06-19T12:09:42Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1803 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[4349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[4349](info): set up 2 sockets
Jun 19 12:09:42 dist-test-slave-c910 krb5kdc[4349](info): commencing operation
krb5kdc: starting...
W20260619 12:09:43.743268 804 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:43 dist-test-slave-c910 krb5kdc[4349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:43.856452 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42505
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:43.935815 4362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:43.936112 4362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:43.936157 4362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:43.937305 4362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:43.937376 4362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:43.937400 4362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:43.937422 4362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:43.937453 4362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:43.938885 4362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:43.939159 4362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:43.939424 4362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:43.944018 4368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:43.944051 4370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:43.944069 4367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:43.944317 4362 server_base.cc:1034] running on GCE node
I20260619 12:09:43.944550 4362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:43.944820 4362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:43.946012 4362 hybrid_clock.cc:648] HybridClock initialized: now 1781870983946003 us; error 37 us; skew 500 ppm
Jun 19 12:09:43 dist-test-slave-c910 krb5kdc[4349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870983, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:43.948486 4362 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:43.948901 4362 webserver.cc:460] Webserver started at http://127.0.201.62:38193/ using document root <none> and password file <none>
I20260619 12:09:43.949126 4362 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:43.949194 4362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:43.949342 4362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:43.950395 4362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "f522f9cfaf8d44aa843062e55ebd7869"
format_stamp: "Formatted at 2026-06-19 12:09:43 on dist-test-slave-c910"
server_key: "954815f4288169b5337e63d739e487c1"
server_key_iv: "a9d9602b2f6e984a2b385a3a2a0aa955"
server_key_version: "encryptionkey@0"
I20260619 12:09:43.950763 4362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "f522f9cfaf8d44aa843062e55ebd7869"
format_stamp: "Formatted at 2026-06-19 12:09:43 on dist-test-slave-c910"
server_key: "954815f4288169b5337e63d739e487c1"
server_key_iv: "a9d9602b2f6e984a2b385a3a2a0aa955"
server_key_version: "encryptionkey@0"
I20260619 12:09:43.952168 4362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:43.952947 4377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:43.953107 4362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:43.953210 4362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "f522f9cfaf8d44aa843062e55ebd7869"
format_stamp: "Formatted at 2026-06-19 12:09:43 on dist-test-slave-c910"
server_key: "954815f4288169b5337e63d739e487c1"
server_key_iv: "a9d9602b2f6e984a2b385a3a2a0aa955"
server_key_version: "encryptionkey@0"
I20260619 12:09:43.953281 4362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:43.973256 4362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:43.975222 4362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:43.975394 4362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:43.978984 4362 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42505
I20260619 12:09:43.979029 4429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42505 every 8 connection(s)
I20260619 12:09:43.979408 4362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:43.980304 4430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:43.981146 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4362
I20260619 12:09:43.981261 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:43.981439 804 external_mini_cluster.cc:1428] Setting key bf623fde02ab439f195449fd13ceadeb
I20260619 12:09:43.983549 4430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869: Bootstrap starting.
I20260619 12:09:43.984282 4430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:43.984580 4430 log.cc:824] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:43.985316 4430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869: No bootstrap required, opened a new log
I20260619 12:09:43.987636 4430 raft_consensus.cc:348] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f522f9cfaf8d44aa843062e55ebd7869" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42505 } }
I20260619 12:09:43.987768 4430 raft_consensus.cc:374] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:43.987790 4430 raft_consensus.cc:729] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f522f9cfaf8d44aa843062e55ebd7869, State: Initialized, Role: FOLLOWER
I20260619 12:09:43.987950 4430 consensus_queue.cc:260] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f522f9cfaf8d44aa843062e55ebd7869" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42505 } }
I20260619 12:09:43.988045 4430 raft_consensus.cc:388] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:43.988070 4430 raft_consensus.cc:482] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:43.988140 4430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 12:09:43 dist-test-slave-c910 krb5kdc[4349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:43.988694 4430 raft_consensus.cc:504] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f522f9cfaf8d44aa843062e55ebd7869" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42505 } }
I20260619 12:09:43.988811 4430 leader_election.cc:302] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f522f9cfaf8d44aa843062e55ebd7869; no voters:
I20260619 12:09:43.988992 4430 leader_election.cc:288] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:43.989030 4435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:43.989185 4435 raft_consensus.cc:686] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [term 1 LEADER]: Becoming Leader. State: Replica: f522f9cfaf8d44aa843062e55ebd7869, State: Running, Role: LEADER
I20260619 12:09:43.989280 4430 sys_catalog.cc:564] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:43.989280 4435 consensus_queue.cc:237] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f522f9cfaf8d44aa843062e55ebd7869" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42505 } }
I20260619 12:09:43.989666 4435 sys_catalog.cc:455] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f522f9cfaf8d44aa843062e55ebd7869. Latest consensus state: current_term: 1 leader_uuid: "f522f9cfaf8d44aa843062e55ebd7869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f522f9cfaf8d44aa843062e55ebd7869" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42505 } } }
I20260619 12:09:43.989758 4435 sys_catalog.cc:458] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:43.989679 4437 sys_catalog.cc:455] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f522f9cfaf8d44aa843062e55ebd7869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f522f9cfaf8d44aa843062e55ebd7869" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42505 } } }
I20260619 12:09:43.989903 4437 sys_catalog.cc:458] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:43.990017 4440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:43.990532 4440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:43.992812 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:43.982007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.0.201.62:42505)
0619 12:09:43.982191 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:43.982201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:43.982233 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:09:43.982736 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:43.982744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:43.982809 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:43.982958 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:43.983637 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:43.984987 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:43.985786 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:43.986400 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:43.986528 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:43.988929 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:43.988934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:43.988934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:43.988947 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:43.991238 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:43.991618 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:43.991621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:43.991621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:43.991733 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:43.991950 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:43.991950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:43.991951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:43.992160 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:43.992305 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:43.992458 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 12:09:43.992610 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260619 12:09:43.992846 4440 catalog_manager.cc:1347] Generated new cluster ID: fdadab7c5779451387c08c733cd6a2d3
I20260619 12:09:43.992880 4440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:44.013587 4440 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:44.014173 4440 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:44.029206 4440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f522f9cfaf8d44aa843062e55ebd7869: Generated new TSK 0
I20260619 12:09:44.029414 4440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:44.059579 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42505
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:44.140730 4456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:44.140933 4456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:44.140959 4456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:44.142124 4456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:44.142166 4456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:44.142194 4456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:44.143857 4456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:44.144099 4456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:44.144335 4456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:44.149411 4461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:44.149428 4464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:44.149456 4462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:44.149459 4456 server_base.cc:1034] running on GCE node
I20260619 12:09:44.149906 4456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:44.150120 4456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:44.151270 4456 hybrid_clock.cc:648] HybridClock initialized: now 1781870984151251 us; error 38 us; skew 500 ppm
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:44.163964 4456 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:44.164453 4456 webserver.cc:460] Webserver started at http://127.0.201.1:45219/ using document root <none> and password file <none>
I20260619 12:09:44.164678 4456 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:44.164723 4456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:44.164822 4456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:44.165892 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "aeb84127b0b24537bd1a315b898e82ac"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "49230674d13fa2b09293834f8b15f0a1"
server_key_iv: "4f68d74a268b36ee8ac089225da5ceb9"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.166220 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "aeb84127b0b24537bd1a315b898e82ac"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "49230674d13fa2b09293834f8b15f0a1"
server_key_iv: "4f68d74a268b36ee8ac089225da5ceb9"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.167623 4456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:44.168352 4471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.168632 4456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:44.168747 4456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "aeb84127b0b24537bd1a315b898e82ac"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "49230674d13fa2b09293834f8b15f0a1"
server_key_iv: "4f68d74a268b36ee8ac089225da5ceb9"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.168826 4456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:44.182058 4456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:44.183847 4456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:44.184006 4456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:44.184226 4456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:44.184518 4456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:44.184571 4456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.184609 4456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:44.184638 4456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.190387 4456 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:40659
I20260619 12:09:44.190482 4584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:40659 every 8 connection(s)
I20260619 12:09:44.190778 4456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:44.194485 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4456
I20260619 12:09:44.194566 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:44.194767 804 external_mini_cluster.cc:1428] Setting key 63092c5efb15889ab8b9a965a13fda8b
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:44.200975 4587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.191470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42505 (local address 127.0.201.1:35741)
0619 12:09:44.191609 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:44.191641 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:44.192528 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:44.192737 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:44.192741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:44.192977 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:44.193155 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:44.193159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.193804 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.193805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:44.194707 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:44.194710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.195251 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.195253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:44.195387 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.195916 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:44.195926 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:44.197789 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:44.199717 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:44.199723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:44.199724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:44.199925 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:44.200157 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:44.200157 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:44.200158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:44.200244 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:44.200532 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:44.200536 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:44.200638 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 12:09:44.200719 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:09:44.200828 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260619 12:09:44.201016 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.191489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:35741 (local address 127.0.201.62:42505)
0619 12:09:44.191531 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:44.191533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:44.191707 (+ 174us) server_negotiation.cc:407] Connection header received
0619 12:09:44.192616 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:44.192617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:44.192627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:44.192685 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:44.193233 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.193721 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.194799 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.195130 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.195221 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.197879 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:44.197880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:44.197880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:44.197888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:44.199641 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.200010 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.200011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.200011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.200086 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.200308 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.200309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.200309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.200461 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:44.200532 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:44.200770 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 12:09:44.200871 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:44.201426 4585 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42505
I20260619 12:09:44.201514 4585 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:44.201694 4585 heartbeater.cc:511] Master 127.0.201.62:42505 requested a full tablet report, sending...
I20260619 12:09:44.202265 4394 ts_manager.cc:194] Registered new tserver with Master: aeb84127b0b24537bd1a315b898e82ac (127.0.201.1:40659)
I20260619 12:09:44.203238 4394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:35741
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:44.220005 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42505
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:44.303037 4590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:44.303303 4590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:44.303342 4590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:44.304482 4590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:44.304555 4590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:44.304601 4590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:44.306035 4590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:44.306291 4590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:44.306527 4590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:44.311446 4596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:44.311450 4595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:44.311514 4590 server_base.cc:1034] running on GCE node
W20260619 12:09:44.311497 4598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:44.311887 4590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:44.312088 4590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:44.313269 4590 hybrid_clock.cc:648] HybridClock initialized: now 1781870984313252 us; error 33 us; skew 500 ppm
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:44.315819 4590 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:44.316295 4590 webserver.cc:460] Webserver started at http://127.0.201.2:38857/ using document root <none> and password file <none>
I20260619 12:09:44.316519 4590 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:44.316584 4590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:44.316741 4590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:44.317718 4590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "1d291709de494b359b46d86596efdd47"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "c2eb0c6850f059a2f4b11340e91e3b51"
server_key_iv: "bbde3100f6dbd766db8161f6f8e4c195"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.318058 4590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "1d291709de494b359b46d86596efdd47"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "c2eb0c6850f059a2f4b11340e91e3b51"
server_key_iv: "bbde3100f6dbd766db8161f6f8e4c195"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.319375 4590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:44.320032 4605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.320194 4590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:44.320286 4590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "1d291709de494b359b46d86596efdd47"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "c2eb0c6850f059a2f4b11340e91e3b51"
server_key_iv: "bbde3100f6dbd766db8161f6f8e4c195"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.320367 4590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:44.327677 4590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:44.329474 4590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:44.329628 4590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:44.329859 4590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:44.330152 4590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:44.330204 4590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.330245 4590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:44.330272 4590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.336081 4590 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:36099
I20260619 12:09:44.336120 4718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:36099 every 8 connection(s)
I20260619 12:09:44.336417 4590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:44.344524 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4590
I20260619 12:09:44.344637 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:44.344843 804 external_mini_cluster.cc:1428] Setting key e8c126427ada7388de9b396ac334117b
I20260619 12:09:44.345865 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.337090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:57437 (local address 127.0.201.62:42505)
0619 12:09:44.337178 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:44.337180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:44.337337 (+ 157us) server_negotiation.cc:407] Connection header received
0619 12:09:44.338349 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:44.338350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:44.338363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:44.338408 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:44.339031 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.339611 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.340597 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.340862 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.340920 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.343066 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:44.343068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:44.343068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:44.343076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:44.344285 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.344784 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.344785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.344785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.344838 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.345113 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.345114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.345114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.345235 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:44.345308 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:44.345626 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 12:09:44.345733 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:09:44.346050 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.337020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42505 (local address 127.0.201.2:57437)
0619 12:09:44.337210 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:44.337252 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:44.338225 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:44.338493 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:44.338496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:44.338746 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:44.338928 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:44.338931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.339709 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.339710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:44.340497 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:44.340499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.340937 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.340938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:44.341063 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.341599 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:44.341607 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:44.342973 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:44.344380 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:44.344386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:44.344388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:44.344711 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:44.344906 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:44.344907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:44.344907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:44.345038 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:44.345313 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:44.345319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:44.345474 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 12:09:44.345590 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 12:09:44.345789 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260619 12:09:44.346496 4719 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42505
I20260619 12:09:44.346583 4719 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:44.346767 4719 heartbeater.cc:511] Master 127.0.201.62:42505 requested a full tablet report, sending...
I20260619 12:09:44.347172 4394 ts_manager.cc:194] Registered new tserver with Master: 1d291709de494b359b46d86596efdd47 (127.0.201.2:36099)
I20260619 12:09:44.347558 4394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:57437
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:44.369238 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42505
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:44.450150 4724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:44.450345 4724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:44.450368 4724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:44.451575 4724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:44.451615 4724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:44.451664 4724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:44.453063 4724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:44.453287 4724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:44.453511 4724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:44.458493 4732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:44.458477 4730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:44.458477 4729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:44.458671 4724 server_base.cc:1034] running on GCE node
I20260619 12:09:44.458910 4724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:44.459141 4724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:44.460321 4724 hybrid_clock.cc:648] HybridClock initialized: now 1781870984460307 us; error 30 us; skew 500 ppm
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:44.462611 4724 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:44.463014 4724 webserver.cc:460] Webserver started at http://127.0.201.3:35187/ using document root <none> and password file <none>
I20260619 12:09:44.463280 4724 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:44.463346 4724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:44.463512 4724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:44.464488 4724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "1ae7aeab0fe14078ada0a39d9ae1c4e5"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "2cef1be08f395ca7f7ecd94ae482ce9a"
server_key_iv: "69005b3417f56bbaaceef91eb90bc249"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.464841 4724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "1ae7aeab0fe14078ada0a39d9ae1c4e5"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "2cef1be08f395ca7f7ecd94ae482ce9a"
server_key_iv: "69005b3417f56bbaaceef91eb90bc249"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.466081 4724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:09:44.466780 4739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.466940 4724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:44.467033 4724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "1ae7aeab0fe14078ada0a39d9ae1c4e5"
format_stamp: "Formatted at 2026-06-19 12:09:44 on dist-test-slave-c910"
server_key: "2cef1be08f395ca7f7ecd94ae482ce9a"
server_key_iv: "69005b3417f56bbaaceef91eb90bc249"
server_key_version: "encryptionkey@0"
I20260619 12:09:44.467116 4724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:44.479995 4724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:44.481858 4724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:44.482015 4724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:44.482239 4724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:44.482543 4724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:44.482596 4724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.482646 4724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:44.482673 4724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:44.488408 4724 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:44733
I20260619 12:09:44.488499 4852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:44733 every 8 connection(s)
I20260619 12:09:44.488780 4724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:44.494024 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4724
I20260619 12:09:44.494119 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:44.494302 804 external_mini_cluster.cc:1428] Setting key 06c531caa513768dddc6f360cea8e4b0
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:44.497996 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.489510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:37519 (local address 127.0.201.62:42505)
0619 12:09:44.489578 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:44.489580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:44.489854 (+ 274us) server_negotiation.cc:407] Connection header received
0619 12:09:44.490885 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:44.490886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:44.490897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:44.491032 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:44.491648 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.492141 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.493249 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.493437 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.493507 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.495770 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:44.495771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:44.495772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:44.495778 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:44.496917 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.497257 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.497258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.497258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.497307 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.497502 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.497502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.497503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.497598 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:44.497647 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:44.497845 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 12:09:44.497900 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:44.498121 4855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.489538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42505 (local address 127.0.201.3:37519)
0619 12:09:44.489716 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:44.489749 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:44.490762 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:44.491070 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:44.491073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:44.491367 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:44.491551 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:44.491555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.492217 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.492217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:44.493171 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:44.493173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.493524 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.493525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:44.493618 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.494133 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:44.494143 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:44.495697 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:44.496982 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:44.496987 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:44.496988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:44.497194 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:44.497362 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:44.497363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:44.497363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:44.497450 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:44.497654 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:44.497657 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:44.497758 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 12:09:44.497835 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:09:44.497943 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260619 12:09:44.498674 4853 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42505
I20260619 12:09:44.498786 4853 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:44.498950 4853 heartbeater.cc:511] Master 127.0.201.62:42505 requested a full tablet report, sending...
I20260619 12:09:44.499344 4394 ts_manager.cc:194] Registered new tserver with Master: 1ae7aeab0fe14078ada0a39d9ae1c4e5 (127.0.201.3:44733)
I20260619 12:09:44.499663 4394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:37519
I20260619 12:09:44.506065 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:44.528033 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.520583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.0.201.1:40659)
0619 12:09:44.520733 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:44.520738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:44.520749 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:44.520776 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:44.520778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:44.520800 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:44.520881 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:44.521443 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.522156 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.522948 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.523313 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.523400 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.525619 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:44.525625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:44.525625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:44.525631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:44.527006 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.527370 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.527373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.527373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.527424 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.527577 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.527578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.527578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.527699 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:44.527793 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:44.527857 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 12:09:44.527916 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:44.535939 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:44.529417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.0.201.62:42505)
0619 12:09:44.529462 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:44.529464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:44.529483 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:09:44.529559 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:44.529560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:44.529569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:44.529623 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:44.530083 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.530570 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.531306 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:44.531507 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:44.531591 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:44.533637 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:44.533638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:44.533639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:44.533646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:44.534839 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.535150 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.535151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.535151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.535261 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:44.535447 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:44.535448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:44.535448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:44.535592 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:44.535669 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:44.535752 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 12:09:44.535813 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 12:09:44.536247 4394 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44650
I20260619 12:09:44.536617 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4456
I20260619 12:09:44.541363 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4590
I20260619 12:09:44.545593 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4724
I20260619 12:09:44.549777 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4362
2026-06-19T12:09:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1844 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4865](info): set up 2 sockets
Jun 19 12:09:44 dist-test-slave-c910 krb5kdc[4865](info): commencing operation
krb5kdc: starting...
W20260619 12:09:45.587186 804 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:45 dist-test-slave-c910 krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:45.698019 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46839
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:46839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:45.779526 4878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:45.779759 4878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:45.779795 4878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:45.780964 4878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:45.781031 4878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:45.781056 4878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:45.781077 4878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:45.781104 4878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:45.782626 4878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:46839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:45.782900 4878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:45.783128 4878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:45.787734 4886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:45.787732 4883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:45.787741 4884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:45.788089 4878 server_base.cc:1034] running on GCE node
I20260619 12:09:45.788339 4878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:45.788625 4878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:45.789783 4878 hybrid_clock.cc:648] HybridClock initialized: now 1781870985789786 us; error 45 us; skew 500 ppm
Jun 19 12:09:45 dist-test-slave-c910 krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:45.792300 4878 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:45.792670 4878 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:45.792740 4878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:45.792893 4878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:45.793968 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "8395171dc1d44c2a90a0d9eab2114e7c"
format_stamp: "Formatted at 2026-06-19 12:09:45 on dist-test-slave-c910"
server_key: "5d1f47add1b3491032fc6fd5278bef7e"
server_key_iv: "889749dc4dcc6d099daa37dce4c89955"
server_key_version: "encryptionkey@0"
I20260619 12:09:45.794332 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "8395171dc1d44c2a90a0d9eab2114e7c"
format_stamp: "Formatted at 2026-06-19 12:09:45 on dist-test-slave-c910"
server_key: "5d1f47add1b3491032fc6fd5278bef7e"
server_key_iv: "889749dc4dcc6d099daa37dce4c89955"
server_key_version: "encryptionkey@0"
I20260619 12:09:45.795673 4878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:45.796370 4892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:45.796537 4878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:45.796633 4878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "8395171dc1d44c2a90a0d9eab2114e7c"
format_stamp: "Formatted at 2026-06-19 12:09:45 on dist-test-slave-c910"
server_key: "5d1f47add1b3491032fc6fd5278bef7e"
server_key_iv: "889749dc4dcc6d099daa37dce4c89955"
server_key_version: "encryptionkey@0"
I20260619 12:09:45.796716 4878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:45.806705 4878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:45.808522 4878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:45.808682 4878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:45.812398 4878 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:46839
I20260619 12:09:45.812492 4944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:46839 every 8 connection(s)
I20260619 12:09:45.812738 4878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:45.813500 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4878
I20260619 12:09:45.813627 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:45.813810 804 external_mini_cluster.cc:1428] Setting key 77356d87fb99633a18d645ff0da1c554
I20260619 12:09:45.813831 4945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:45.817262 4945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c: Bootstrap starting.
I20260619 12:09:45.817950 4945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:45.818230 4945 log.cc:824] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:45.818961 4945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c: No bootstrap required, opened a new log
Jun 19 12:09:45 dist-test-slave-c910 krb5kdc[4865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:45.821189 4945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46839 } }
I20260619 12:09:45.821372 4945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:45.821439 4945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8395171dc1d44c2a90a0d9eab2114e7c, State: Initialized, Role: FOLLOWER
I20260619 12:09:45.821569 4945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46839 } }
I20260619 12:09:45.821645 4945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:45.821699 4945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:45.821748 4945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:45.822484 4945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46839 } }
I20260619 12:09:45.822590 4945 leader_election.cc:302] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8395171dc1d44c2a90a0d9eab2114e7c; no voters:
I20260619 12:09:45.822757 4945 leader_election.cc:288] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:45.822849 4950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:45.823031 4945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:45.823052 4950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [term 1 LEADER]: Becoming Leader. State: Replica: 8395171dc1d44c2a90a0d9eab2114e7c, State: Running, Role: LEADER
I20260619 12:09:45.823179 4950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46839 } }
I20260619 12:09:45.823570 4951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46839 } } }
I20260619 12:09:45.823585 4952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8395171dc1d44c2a90a0d9eab2114e7c. Latest consensus state: current_term: 1 leader_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8395171dc1d44c2a90a0d9eab2114e7c" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46839 } } }
I20260619 12:09:45.823691 4951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:45.823743 4952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:45.824453 4954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:45.824967 4954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:45.825467 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:45.814367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.0.201.62:46839)
0619 12:09:45.814530 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:45.814539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:45.814579 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:09:45.814968 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:45.814974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:45.815019 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:45.815132 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:45.815980 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:45.816839 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:45.817854 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:45.818302 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:45.818422 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:45.821076 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:45.821080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:45.821081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:45.821102 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:45.823348 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:45.823846 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:45.823849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:45.823849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:45.823983 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:45.824225 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:45.824226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:45.824226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:45.824542 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:45.825071 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:45.825142 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 12:09:45.825275 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260619 12:09:45.826499 4954 catalog_manager.cc:1347] Generated new cluster ID: d79abbe516264c189a1508dd4a9c554f
I20260619 12:09:45.826560 4954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:45.857487 4954 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:45.858058 4954 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:45.862375 4954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c: Generated new TSK 0
I20260619 12:09:45.862565 4954 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:45.890362 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46839
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:45.970669 4971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:45.970867 4971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:45.970890 4971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:45.972095 4971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:45.972136 4971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:45.972163 4971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:45.973655 4971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:45.973897 4971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:45.974153 4971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:45.978966 4976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:45.978997 4977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:45.978957 4979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:45.979384 4971 server_base.cc:1034] running on GCE node
I20260619 12:09:45.979614 4971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:45.979884 4971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:45.981019 4971 hybrid_clock.cc:648] HybridClock initialized: now 1781870985981007 us; error 28 us; skew 500 ppm
Jun 19 12:09:45 dist-test-slave-c910 krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:45.983457 4971 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:45.983851 4971 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:45.983933 4971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:45.984083 4971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:45.985235 4971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "ac57b5a839754dc68898e41ad010a465"
format_stamp: "Formatted at 2026-06-19 12:09:45 on dist-test-slave-c910"
server_key: "0cf45149c3994368d28e7d0a2b3804bf"
server_key_iv: "2fd65cce3a34484a41fc7e505b351cc7"
server_key_version: "encryptionkey@0"
I20260619 12:09:45.985594 4971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "ac57b5a839754dc68898e41ad010a465"
format_stamp: "Formatted at 2026-06-19 12:09:45 on dist-test-slave-c910"
server_key: "0cf45149c3994368d28e7d0a2b3804bf"
server_key_iv: "2fd65cce3a34484a41fc7e505b351cc7"
server_key_version: "encryptionkey@0"
I20260619 12:09:45.986922 4971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:45.987807 4985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:45.988020 4971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:45.988125 4971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "ac57b5a839754dc68898e41ad010a465"
format_stamp: "Formatted at 2026-06-19 12:09:45 on dist-test-slave-c910"
server_key: "0cf45149c3994368d28e7d0a2b3804bf"
server_key_iv: "2fd65cce3a34484a41fc7e505b351cc7"
server_key_version: "encryptionkey@0"
I20260619 12:09:45.988212 4971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.004130 4971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.006333 4971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.006512 4971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.006744 4971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:46.007059 4971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:46.007128 4971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.007169 4971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:46.007272 4971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.013229 4971 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:41451
I20260619 12:09:46.013263 5098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:41451 every 8 connection(s)
I20260619 12:09:46.013614 4971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:46.015518 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 4971
I20260619 12:09:46.015609 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:46.015789 804 external_mini_cluster.cc:1428] Setting key 26de7b63e9b36942f8a4572001122e95
Jun 19 12:09:46 dist-test-slave-c910 krb5kdc[4865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
I20260619 12:09:46.024405 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.014409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:46647 (local address 127.0.201.62:46839)
0619 12:09:46.014504 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.014507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.014719 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:09:46.015210 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.015211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.015222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.015275 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.016512 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.017266 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.018617 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.018820 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.018880 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.021355 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.021357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.021357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:46.021365 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.022843 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.023298 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.023299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.023299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.023357 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.023677 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.023678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.023678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.023804 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.023888 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.024253 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.024336 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:09:46.024667 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.014391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46839 (local address 127.0.201.1:46647)
0619 12:09:46.014560 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.014602 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.015073 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.015352 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.015356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.015662 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.016373 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.016380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.017356 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.017357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.018523 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.018527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.018913 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.018914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.019039 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.019641 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:46.019650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.021254 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.022945 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:46.022952 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:46.022954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:46.023185 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:46.023454 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:46.023455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:46.023456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:46.023604 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:46.023910 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.023918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.024107 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.024231 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.024402 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":108,"threads_started":1}
I20260619 12:09:46.025236 5099 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46839
I20260619 12:09:46.025331 5099 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:46.025451 5099 heartbeater.cc:511] Master 127.0.201.62:46839 requested a full tablet report, sending...
I20260619 12:09:46.025976 4909 ts_manager.cc:194] Registered new tserver with Master: ac57b5a839754dc68898e41ad010a465 (127.0.201.1:41451)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
I20260619 12:09:46.026803 4909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:46647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:46.041805 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46839
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:46.126401 5104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:46.126600 5104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:46.126622 5104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:46.127803 5104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:46.127842 5104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:46.127890 5104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:46.129312 5104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:46.129545 5104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:46.129760 5104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:46.134509 5112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.134557 5109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.134511 5110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.134756 5104 server_base.cc:1034] running on GCE node
I20260619 12:09:46.134969 5104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:46.135157 5104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:46.136343 5104 hybrid_clock.cc:648] HybridClock initialized: now 1781870986136329 us; error 29 us; skew 500 ppm
Jun 19 12:09:46 dist-test-slave-c910 krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:46.138741 5104 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:46.139093 5104 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:46.139142 5104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:46.139283 5104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:46.140225 5104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "41f26e8ede0b433e9fdb8e43056f03ba"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "e644316813aeacbd4b6daa0a7e734c3a"
server_key_iv: "7bcc82c716f5b2e43df5eb2bf30d7a6c"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.140518 5104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "41f26e8ede0b433e9fdb8e43056f03ba"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "e644316813aeacbd4b6daa0a7e734c3a"
server_key_iv: "7bcc82c716f5b2e43df5eb2bf30d7a6c"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.141659 5104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:09:46.142278 5118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.142477 5104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:46.142582 5104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "41f26e8ede0b433e9fdb8e43056f03ba"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "e644316813aeacbd4b6daa0a7e734c3a"
server_key_iv: "7bcc82c716f5b2e43df5eb2bf30d7a6c"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.142629 5104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.164371 5104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.166196 5104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.166308 5104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.166563 5104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:46.166829 5104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:46.166859 5104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.166882 5104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:46.166893 5104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.172627 5104 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:34727
I20260619 12:09:46.172669 5231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:34727 every 8 connection(s)
I20260619 12:09:46.172922 5104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:46.176713 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5104
I20260619 12:09:46.176800 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:46.177014 804 external_mini_cluster.cc:1428] Setting key cc6e1b42398486976147802054596610
Jun 19 12:09:46 dist-test-slave-c910 krb5kdc[4865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:46.182904 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.173589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:47679 (local address 127.0.201.62:46839)
0619 12:09:46.173635 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.173637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.173812 (+ 175us) server_negotiation.cc:407] Connection header received
0619 12:09:46.174329 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.174331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.174343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.174406 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.175368 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.175874 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.176907 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.177150 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.177236 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.179597 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.179599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.179599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:46.179606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.181222 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.181814 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.181815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.181816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.181877 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.182127 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.182128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.182128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.182285 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.182362 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.182672 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.182764 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:46.182930 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.173544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46839 (local address 127.0.201.2:47679)
0619 12:09:46.173678 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.173726 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.174222 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.174477 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.174481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.174756 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.175264 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.175269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.175966 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.175967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.176834 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.176837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.177250 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.177251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.177416 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.177989 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:46.178006 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.179494 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.181322 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:46.181328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:46.181331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:46.181708 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:46.181944 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:46.181944 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:46.181944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:46.182057 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:46.182381 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.182386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.182526 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.182626 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.182752 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260619 12:09:46.183740 5232 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46839
I20260619 12:09:46.183847 5232 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:46.184002 5232 heartbeater.cc:511] Master 127.0.201.62:46839 requested a full tablet report, sending...
I20260619 12:09:46.184418 4909 ts_manager.cc:194] Registered new tserver with Master: 41f26e8ede0b433e9fdb8e43056f03ba (127.0.201.2:34727)
I20260619 12:09:46.184979 4909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:47679
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:46.201930 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46839
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:46.281728 5237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:46.281929 5237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:46.281951 5237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:46.283084 5237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:46.283126 5237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:46.283152 5237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:46.284611 5237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:46.284849 5237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:46.285076 5237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:46.289916 5245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.289990 5242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.290033 5243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.290350 5237 server_base.cc:1034] running on GCE node
I20260619 12:09:46.290555 5237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:46.290756 5237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:46.291926 5237 hybrid_clock.cc:648] HybridClock initialized: now 1781870986291912 us; error 29 us; skew 500 ppm
Jun 19 12:09:46 dist-test-slave-c910 krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:46.294409 5237 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:46.294777 5237 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:46.294827 5237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:46.295012 5237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:46.296123 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "1d895f62296f4b9db7265bbddf115dab"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "23b8b7da523ee16b86344f73b90b0233"
server_key_iv: "9f3f4910487c286f2e5d06cbc48ba48d"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.296473 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "1d895f62296f4b9db7265bbddf115dab"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "23b8b7da523ee16b86344f73b90b0233"
server_key_iv: "9f3f4910487c286f2e5d06cbc48ba48d"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.297719 5237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:46.298527 5251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.298722 5237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:46.298826 5237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "1d895f62296f4b9db7265bbddf115dab"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "23b8b7da523ee16b86344f73b90b0233"
server_key_iv: "9f3f4910487c286f2e5d06cbc48ba48d"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.298911 5237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.311741 5237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.314116 5237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.314280 5237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.314514 5237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:46.314852 5237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:46.314908 5237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.314944 5237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:46.314971 5237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.320776 5237 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:46135
I20260619 12:09:46.320811 5364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:46135 every 8 connection(s)
I20260619 12:09:46.321067 5237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:46.326853 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5237
I20260619 12:09:46.326985 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:46.327185 804 external_mini_cluster.cc:1428] Setting key 09929df07814cb41ac1e655993212819
Jun 19 12:09:46 dist-test-slave-c910 krb5kdc[4865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:46.330652 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.321779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:41341 (local address 127.0.201.62:46839)
0619 12:09:46.321844 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.321846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.322098 (+ 252us) server_negotiation.cc:407] Connection header received
0619 12:09:46.322612 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.322613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.322625 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.322692 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.323858 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.324544 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.325703 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.325915 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.325991 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.328271 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.328271 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.328272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:46.328278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.329476 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.329861 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.329862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.329862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.329912 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.330129 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.330130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.330130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.330231 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.330282 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.330527 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.330590 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:09:46.330951 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.321746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46839 (local address 127.0.201.3:41341)
0619 12:09:46.321923 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.322012 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.322507 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.322771 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.322775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.323034 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.323628 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.323633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.324629 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.324630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.325584 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.325586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.326007 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.326008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.326108 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.326585 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:46.326593 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.328181 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.329549 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:46.329555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:46.329556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:46.329794 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:46.329974 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:46.329975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:46.329975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:46.330071 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:46.330287 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.330290 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.330408 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.330496 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.330631 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260619 12:09:46.331591 5365 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46839
I20260619 12:09:46.331704 5365 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:46.331878 5365 heartbeater.cc:511] Master 127.0.201.62:46839 requested a full tablet report, sending...
I20260619 12:09:46.332255 4909 ts_manager.cc:194] Registered new tserver with Master: 1d895f62296f4b9db7265bbddf115dab (127.0.201.3:46135)
I20260619 12:09:46.332679 4909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:41341
I20260619 12:09:46.339123 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:09:46.346494 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.340460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.0.201.62:46839)
0619 12:09:46.340518 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.340520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.340642 (+ 122us) server_negotiation.cc:407] Connection header received
0619 12:09:46.340730 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.340731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.340741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.340810 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.341378 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.341851 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.342563 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.342751 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.342831 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.344054 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.344056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.344056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:46.344063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.345353 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.345698 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.345699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.345700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.345759 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:46.345950 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:46.345951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:46.345952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:46.346106 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.346209 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.346316 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.346393 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:09:46.348229 4909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:46.349797 4909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:46.356328 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.351945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:46135 (local address 127.0.0.1:32792)
0619 12:09:46.352551 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.352560 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.352599 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.352960 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.352961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.352963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:46.353016 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.353017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.354254 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.354256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.355255 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.355258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.356016 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.356017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.356092 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.356106 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.356200 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.356253 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":4,"thread_start_us":69,"threads_started":1}
I20260619 12:09:46.356340 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.352195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.0.201.3:46135)
0619 12:09:46.352707 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.352711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.352721 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:46.352763 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.352765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.352780 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.352853 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:46.353102 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.354129 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.355336 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.355905 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.356012 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.356076 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.356216 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.356253 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":68,"threads_started":1}
I20260619 12:09:46.356634 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.351937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:34727 (local address 127.0.0.1:36250)
0619 12:09:46.352315 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.352334 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.352414 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.352747 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.352748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.352749 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:46.352842 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.352844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.353989 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.353990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.355456 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.355459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.356420 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.356420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.356475 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.356479 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.356525 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.356563 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":32,"thread_start_us":80,"threads_started":1}
I20260619 12:09:46.356874 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.352116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.0.201.2:34727)
0619 12:09:46.352260 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.352266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.352409 (+ 143us) server_negotiation.cc:407] Connection header received
0619 12:09:46.352578 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.352580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.352597 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.352665 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:46.352927 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.353884 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.355547 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.356265 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.356334 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.356385 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.356741 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.356787 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260619 12:09:46.357350 5299 tablet_service.cc:1467] Processing CreateTablet for tablet b9a9e903c0994fa5b850c1e6ef2cb66b (DEFAULT_TABLE table=test-table [id=6ed6cf9af5ad4602b80c508c1e8c0ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:46.357484 5166 tablet_service.cc:1467] Processing CreateTablet for tablet b9a9e903c0994fa5b850c1e6ef2cb66b (DEFAULT_TABLE table=test-table [id=6ed6cf9af5ad4602b80c508c1e8c0ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:46.357707 5299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a9e903c0994fa5b850c1e6ef2cb66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.357728 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.351937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:41451 (local address 127.0.0.1:59322)
0619 12:09:46.352124 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.352333 (+ 209us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.352414 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.352743 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.352746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.352749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:46.352842 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.352844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.355258 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.355259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.356357 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.356360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.357491 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.357492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.357555 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.357567 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.357623 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.357660 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260619 12:09:46.357746 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.352148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.0.201.1:41451)
0619 12:09:46.352308 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.352312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.352408 (+ 96us) server_negotiation.cc:407] Connection header received
0619 12:09:46.352494 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.352496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.352526 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.352625 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:46.354005 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.355130 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.356442 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.357388 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.357487 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.357538 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.357636 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.357671 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260619 12:09:46.357748 5166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a9e903c0994fa5b850c1e6ef2cb66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.360997 5380 tablet_bootstrap.cc:492] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Bootstrap starting.
I20260619 12:09:46.361541 5033 tablet_service.cc:1467] Processing CreateTablet for tablet b9a9e903c0994fa5b850c1e6ef2cb66b (DEFAULT_TABLE table=test-table [id=6ed6cf9af5ad4602b80c508c1e8c0ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:46.361690 5380 tablet_bootstrap.cc:654] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.361850 5033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a9e903c0994fa5b850c1e6ef2cb66b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.362231 5380 log.cc:824] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.362550 5379 tablet_bootstrap.cc:492] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Bootstrap starting.
I20260619 12:09:46.362980 5380 tablet_bootstrap.cc:492] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: No bootstrap required, opened a new log
I20260619 12:09:46.363049 5380 ts_tablet_manager.cc:1397] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:46.363328 5379 tablet_bootstrap.cc:654] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.363845 5379 log.cc:824] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.364523 5379 tablet_bootstrap.cc:492] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: No bootstrap required, opened a new log
I20260619 12:09:46.364609 5379 ts_tablet_manager.cc:1397] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:09:46.365171 5383 tablet_bootstrap.cc:492] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: Bootstrap starting.
I20260619 12:09:46.365099 5380 raft_consensus.cc:348] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.365260 5380 raft_consensus.cc:374] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.365311 5380 raft_consensus.cc:729] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f26e8ede0b433e9fdb8e43056f03ba, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.365432 5380 consensus_queue.cc:260] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.365728 5232 heartbeater.cc:503] Master 127.0.201.62:46839 was elected leader, sending a full tablet report...
I20260619 12:09:46.365726 5380 ts_tablet_manager.cc:1428] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:09:46.365890 5383 tablet_bootstrap.cc:654] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.366464 5383 log.cc:824] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.366628 5379 raft_consensus.cc:348] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.366814 5379 raft_consensus.cc:374] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.366869 5379 raft_consensus.cc:729] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d895f62296f4b9db7265bbddf115dab, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.366987 5379 consensus_queue.cc:260] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.367234 5383 tablet_bootstrap.cc:492] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: No bootstrap required, opened a new log
I20260619 12:09:46.367298 5383 ts_tablet_manager.cc:1397] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:46.367343 5365 heartbeater.cc:503] Master 127.0.201.62:46839 was elected leader, sending a full tablet report...
I20260619 12:09:46.367595 5379 ts_tablet_manager.cc:1428] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:09:46.368975 5383 raft_consensus.cc:348] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.369100 5383 raft_consensus.cc:374] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.369122 5383 raft_consensus.cc:729] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac57b5a839754dc68898e41ad010a465, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.369252 5383 consensus_queue.cc:260] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.369519 5383 ts_tablet_manager.cc:1428] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:46.369532 5099 heartbeater.cc:503] Master 127.0.201.62:46839 was elected leader, sending a full tablet report...
I20260619 12:09:46.369673 5384 raft_consensus.cc:482] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:46.369733 5384 raft_consensus.cc:504] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.370091 5384 leader_election.cc:288] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac57b5a839754dc68898e41ad010a465 (127.0.201.1:41451), 1d895f62296f4b9db7265bbddf115dab (127.0.201.3:46135)
I20260619 12:09:46.373915 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.370250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54007 (local address 127.0.201.3:46135)
0619 12:09:46.370289 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.370291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.370588 (+ 297us) server_negotiation.cc:407] Connection header received
0619 12:09:46.370610 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.370610 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.370619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.370673 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:46.370815 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.371502 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.373000 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.373552 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.373625 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.373636 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.373818 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.373852 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:46.373915 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.370378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:41547 (local address 127.0.201.1:41451)
0619 12:09:46.370438 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.370440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.370446 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:09:46.370468 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.370469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.370478 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.370557 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:46.370694 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.371459 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.373000 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.373550 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.373630 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.373638 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.373818 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.373852 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:09:46.373939 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.370212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:41451 (local address 127.0.201.2:41547)
0619 12:09:46.370381 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.370390 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.370419 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.370560 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.370561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.370563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:46.370618 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.370620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.371566 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.371567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.372897 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.372900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.373647 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.373648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.373734 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.373736 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.373807 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.373860 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":83,"thread_start_us":78,"threads_started":1}
I20260619 12:09:46.374118 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.370212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:46135 (local address 127.0.201.2:54007)
0619 12:09:46.370342 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.370354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.370418 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.370677 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.370678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.370680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:46.370752 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.370753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.371572 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.371573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.372900 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:46.372901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.373647 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.373648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:46.373731 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.373736 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.373814 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.373860 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":38}
I20260619 12:09:46.374394 5319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9e903c0994fa5b850c1e6ef2cb66b" candidate_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d895f62296f4b9db7265bbddf115dab" is_pre_election: true
I20260619 12:09:46.374414 5053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9e903c0994fa5b850c1e6ef2cb66b" candidate_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac57b5a839754dc68898e41ad010a465" is_pre_election: true
I20260619 12:09:46.374534 5319 raft_consensus.cc:2454] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f26e8ede0b433e9fdb8e43056f03ba in term 0.
I20260619 12:09:46.374552 5053 raft_consensus.cc:2454] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f26e8ede0b433e9fdb8e43056f03ba in term 0.
I20260619 12:09:46.374734 5121 leader_election.cc:302] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d895f62296f4b9db7265bbddf115dab, 41f26e8ede0b433e9fdb8e43056f03ba; no voters:
I20260619 12:09:46.374872 5384 raft_consensus.cc:2781] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:46.374924 5384 raft_consensus.cc:482] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:46.374946 5384 raft_consensus.cc:3037] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:46.375634 5384 raft_consensus.cc:504] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.375774 5384 leader_election.cc:288] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [CANDIDATE]: Term 1 election: Requested vote from peers ac57b5a839754dc68898e41ad010a465 (127.0.201.1:41451), 1d895f62296f4b9db7265bbddf115dab (127.0.201.3:46135)
I20260619 12:09:46.375996 5319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9e903c0994fa5b850c1e6ef2cb66b" candidate_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d895f62296f4b9db7265bbddf115dab"
I20260619 12:09:46.376003 5053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9e903c0994fa5b850c1e6ef2cb66b" candidate_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac57b5a839754dc68898e41ad010a465"
I20260619 12:09:46.376065 5319 raft_consensus.cc:3037] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:46.376094 5053 raft_consensus.cc:3037] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:46.376775 5319 raft_consensus.cc:2454] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f26e8ede0b433e9fdb8e43056f03ba in term 1.
I20260619 12:09:46.376792 5053 raft_consensus.cc:2454] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f26e8ede0b433e9fdb8e43056f03ba in term 1.
I20260619 12:09:46.376971 5119 leader_election.cc:302] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1d895f62296f4b9db7265bbddf115dab, 41f26e8ede0b433e9fdb8e43056f03ba, ac57b5a839754dc68898e41ad010a465; no voters:
I20260619 12:09:46.377085 5384 raft_consensus.cc:2781] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:46.377254 5384 raft_consensus.cc:686] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 1 LEADER]: Becoming Leader. State: Replica: 41f26e8ede0b433e9fdb8e43056f03ba, State: Running, Role: LEADER
I20260619 12:09:46.377355 5384 consensus_queue.cc:237] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } }
I20260619 12:09:46.378227 4909 catalog_manager.cc:5495] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 41f26e8ede0b433e9fdb8e43056f03ba (127.0.201.2). New cstate: current_term: 1 leader_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41f26e8ede0b433e9fdb8e43056f03ba" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:46.382982 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.380639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.0.201.2:34727)
0619 12:09:46.380700 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.380702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.380711 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:46.380786 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.380787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.380804 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.380871 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:46.380990 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.381447 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.382215 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:46.382415 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:46.382496 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:46.382665 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:46.382765 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:46.382855 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.382874 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.382892 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:09:46.384621 5319 raft_consensus.cc:1261] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 1 FOLLOWER]: Refusing update from remote peer 41f26e8ede0b433e9fdb8e43056f03ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:46.384653 5053 raft_consensus.cc:1261] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 1 FOLLOWER]: Refusing update from remote peer 41f26e8ede0b433e9fdb8e43056f03ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:46.384948 5389 consensus_queue.cc:1035] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d895f62296f4b9db7265bbddf115dab" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:46.385028 5384 consensus_queue.cc:1035] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac57b5a839754dc68898e41ad010a465" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 41451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:46.387478 5393 mvcc.cc:205] Tried to move back new op lower bound from 7298543560229183488 to 7298543560202575872. Current Snapshot: MvccSnapshot[applied={T|T < 7298543560229183488}]
I20260619 12:09:46.387864 5394 mvcc.cc:205] Tried to move back new op lower bound from 7298543560229183488 to 7298543560202575872. Current Snapshot: MvccSnapshot[applied={T|T < 7298543560229183488}]
I20260619 12:09:46.389261 4908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:46.389374 4908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:46.390659 4908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8395171dc1d44c2a90a0d9eab2114e7c: Sending DeleteTablet for 3 replicas of tablet b9a9e903c0994fa5b850c1e6ef2cb66b
I20260619 12:09:46.391001 5033 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a9e903c0994fa5b850c1e6ef2cb66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:46 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:59322
I20260619 12:09:46.391067 5166 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a9e903c0994fa5b850c1e6ef2cb66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:46 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36250
I20260619 12:09:46.391268 5402 tablet_replica.cc:330] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465: stopping tablet replica
I20260619 12:09:46.391352 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4971
I20260619 12:09:46.391360 5402 raft_consensus.cc:2229] T b9a9e903c0994fa5b850c1e6ef2cb66b P ac57b5a839754dc68898e41ad010a465 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:46.392782 5403 tablet_replica.cc:330] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: stopping tablet replica
I20260619 12:09:46.392874 5403 raft_consensus.cc:2229] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:46.392978 5403 raft_consensus.cc:2258] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:46.393357 5403 ts_tablet_manager.cc:1905] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:46.393893 5299 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a9e903c0994fa5b850c1e6ef2cb66b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:46 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:32792
I20260619 12:09:46.394980 5403 ts_tablet_manager.cc:1918] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:46.395051 5403 log.cc:1196] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/b9a9e903c0994fa5b850c1e6ef2cb66b
I20260619 12:09:46.395311 5404 tablet_replica.cc:330] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: stopping tablet replica
I20260619 12:09:46.395346 5403 ts_tablet_manager.cc:1939] T b9a9e903c0994fa5b850c1e6ef2cb66b P 41f26e8ede0b433e9fdb8e43056f03ba: Deleting consensus metadata
I20260619 12:09:46.395500 5404 raft_consensus.cc:2229] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:46.395646 5404 raft_consensus.cc:2258] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:46.395733 4896 catalog_manager.cc:4841] TS 41f26e8ede0b433e9fdb8e43056f03ba (127.0.201.2:34727): tablet b9a9e903c0994fa5b850c1e6ef2cb66b (table test-table [id=6ed6cf9af5ad4602b80c508c1e8c0ce9]) successfully deleted
I20260619 12:09:46.396174 5404 ts_tablet_manager.cc:1905] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:09:46.396293 4893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:41451 (error 108)
I20260619 12:09:46.396461 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5104
I20260619 12:09:46.398730 5404 ts_tablet_manager.cc:1918] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:46.398805 5404 log.cc:1196] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/b9a9e903c0994fa5b850c1e6ef2cb66b
I20260619 12:09:46.399132 5404 ts_tablet_manager.cc:1939] T b9a9e903c0994fa5b850c1e6ef2cb66b P 1d895f62296f4b9db7265bbddf115dab: Deleting consensus metadata
I20260619 12:09:46.399533 4895 catalog_manager.cc:4841] TS 1d895f62296f4b9db7265bbddf115dab (127.0.201.3:46135): tablet b9a9e903c0994fa5b850c1e6ef2cb66b (table test-table [id=6ed6cf9af5ad4602b80c508c1e8c0ce9]) successfully deleted
I20260619 12:09:46.402631 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5237
I20260619 12:09:46.403184 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.402743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:41451 (local address 127.0.0.1:59334)
0619 12:09:46.403109 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.403145 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
W20260619 12:09:46.403396 4893 catalog_manager.cc:4584] TS ac57b5a839754dc68898e41ad010a465 (127.0.201.1:41451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9a9e903c0994fa5b850c1e6ef2cb66b: Network error: Client connection negotiation failed: client connection to 127.0.201.1:41451: connect: Connection refused (error 111)
I20260619 12:09:46.407354 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 4878
2026-06-19T12:09:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1858 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T12:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:46Z Disabled control of system clock
I20260619 12:09:46.435034 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42459
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:09:46.514523 5409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:46.514734 5409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:46.514755 5409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:46.515873 5409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:46.515918 5409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:46.515931 5409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:46.515944 5409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:46.515956 5409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:46.517195 5409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42459
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:46.517373 5409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:46.517587 5409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:09:46.522356 5409 server_base.cc:1034] running on GCE node
W20260619 12:09:46.522320 5415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.522320 5414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.522331 5417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.522784 5409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:46.523002 5409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:46.524134 5409 hybrid_clock.cc:648] HybridClock initialized: now 1781870986524116 us; error 34 us; skew 500 ppm
I20260619 12:09:46.525243 5409 webserver.cc:460] Webserver started at http://127.0.201.62:43915/ using document root <none> and password file <none>
I20260619 12:09:46.525442 5409 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:46.525481 5409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:46.525569 5409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:46.526585 5409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "1a32a9c8ffc944349b3615a7aee3a657"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "f4e78fb12a43ca5eba65e0f974092e3f"
server_key_iv: "d69ab7974c7b07bd11704bfc2bbe40a8"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.526878 5409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "1a32a9c8ffc944349b3615a7aee3a657"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "f4e78fb12a43ca5eba65e0f974092e3f"
server_key_iv: "d69ab7974c7b07bd11704bfc2bbe40a8"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.528342 5409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:46.529098 5423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.529289 5409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:46.529372 5409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "1a32a9c8ffc944349b3615a7aee3a657"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "f4e78fb12a43ca5eba65e0f974092e3f"
server_key_iv: "d69ab7974c7b07bd11704bfc2bbe40a8"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.529433 5409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.532661 5409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.532864 5409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.532960 5409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.536476 5409 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42459
I20260619 12:09:46.536525 5475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42459 every 8 connection(s)
I20260619 12:09:46.536757 5409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:46.537338 5476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.539124 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5409
I20260619 12:09:46.539281 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:46.539479 804 external_mini_cluster.cc:1428] Setting key decda59b0069e074904fcad35e230415
I20260619 12:09:46.541025 5476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657: Bootstrap starting.
I20260619 12:09:46.541674 5476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.541972 5476 log.cc:824] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.542284 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.540058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.0.201.62:42459)
0619 12:09:46.540312 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.540321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.540350 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:09:46.540846 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.540855 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.540908 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.541062 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.541796 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.541799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.541800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.541804 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.541851 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.541867 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.541976 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.542043 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.542102 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260619 12:09:46.542830 5476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657: No bootstrap required, opened a new log
I20260619 12:09:46.544193 5476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a32a9c8ffc944349b3615a7aee3a657" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42459 } }
I20260619 12:09:46.544322 5476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.544343 5476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a32a9c8ffc944349b3615a7aee3a657, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.544454 5476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a32a9c8ffc944349b3615a7aee3a657" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42459 } }
I20260619 12:09:46.544534 5476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:46.544556 5476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:46.544585 5476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:46.545336 5476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a32a9c8ffc944349b3615a7aee3a657" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42459 } }
I20260619 12:09:46.545444 5476 leader_election.cc:302] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a32a9c8ffc944349b3615a7aee3a657; no voters:
I20260619 12:09:46.545626 5476 leader_election.cc:288] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:46.545657 5481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:46.545763 5481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [term 1 LEADER]: Becoming Leader. State: Replica: 1a32a9c8ffc944349b3615a7aee3a657, State: Running, Role: LEADER
I20260619 12:09:46.545841 5481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a32a9c8ffc944349b3615a7aee3a657" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42459 } }
I20260619 12:09:46.545902 5476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:46.546146 5481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a32a9c8ffc944349b3615a7aee3a657. Latest consensus state: current_term: 1 leader_uuid: "1a32a9c8ffc944349b3615a7aee3a657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a32a9c8ffc944349b3615a7aee3a657" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42459 } } }
I20260619 12:09:46.546226 5481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:46.546159 5483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a32a9c8ffc944349b3615a7aee3a657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a32a9c8ffc944349b3615a7aee3a657" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42459 } } }
I20260619 12:09:46.546280 5483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:46.546403 5485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:46.546882 5485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:46.548568 5485 catalog_manager.cc:1347] Generated new cluster ID: 3961586fe64743f88bb8927983e2c633
I20260619 12:09:46.548622 5485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:46.559015 5485 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:46.559198 5485 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:46.568555 5485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657: Generated new TSK 0
I20260619 12:09:46.568773 5485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:46.575971 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42459
--builtin_ntp_servers=127.0.201.20:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:09:46.676411 5500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:46.676620 5500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:46.676642 5500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:46.677802 5500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:46.677841 5500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:46.677867 5500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:46.679219 5500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:46.679478 5500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:46.679786 5500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:46.685426 5508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.685433 5506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.685456 5505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.685814 5500 server_base.cc:1034] running on GCE node
I20260619 12:09:46.686000 5500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:46.686251 5500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:46.687408 5500 hybrid_clock.cc:648] HybridClock initialized: now 1781870986687389 us; error 36 us; skew 500 ppm
I20260619 12:09:46.688629 5500 webserver.cc:460] Webserver started at http://127.0.201.1:44355/ using document root <none> and password file <none>
I20260619 12:09:46.688867 5500 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:46.688938 5500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:46.689101 5500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:46.690146 5500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "3918f4be04bd4773926a3a84e6ddce7d"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "1de3a45e9ec35cbe8aeb320b59c39f43"
server_key_iv: "56b617c5abea6a988d2f9ba6a1726f1f"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.690492 5500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "3918f4be04bd4773926a3a84e6ddce7d"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "1de3a45e9ec35cbe8aeb320b59c39f43"
server_key_iv: "56b617c5abea6a988d2f9ba6a1726f1f"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.691901 5500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:46.692688 5514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.692862 5500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:46.692958 5500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "3918f4be04bd4773926a3a84e6ddce7d"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "1de3a45e9ec35cbe8aeb320b59c39f43"
server_key_iv: "56b617c5abea6a988d2f9ba6a1726f1f"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.693033 5500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.696298 5500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.696550 5500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.696694 5500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.696892 5500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:46.697189 5500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:46.697242 5500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.697283 5500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:46.697309 5500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.703083 5500 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:40727
I20260619 12:09:46.703148 5627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:40727 every 8 connection(s)
I20260619 12:09:46.703490 5500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:46.706700 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.704280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:53943 (local address 127.0.201.62:42459)
0619 12:09:46.704337 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.704339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.704592 (+ 253us) server_negotiation.cc:407] Connection header received
0619 12:09:46.705068 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.705069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.705075 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.705123 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.706358 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.706360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.706361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.706366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.706381 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.706385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.706453 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.706601 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.706617 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:09:46.706965 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.704269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42459 (local address 127.0.201.1:53943)
0619 12:09:46.704466 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.704499 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.704965 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.705195 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.705198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.705205 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.706151 (+ 946us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:46.706152 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.706180 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:46.706229 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.706477 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.706483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.706486 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.706587 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.706649 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260619 12:09:46.707757 5628 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42459
I20260619 12:09:46.707844 5628 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:46.707993 5628 heartbeater.cc:511] Master 127.0.201.62:42459 requested a full tablet report, sending...
I20260619 12:09:46.708464 5440 ts_manager.cc:194] Registered new tserver with Master: 3918f4be04bd4773926a3a84e6ddce7d (127.0.201.1:40727)
I20260619 12:09:46.711350 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5500
I20260619 12:09:46.711442 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:46.711578 804 external_mini_cluster.cc:1428] Setting key 37c98e74b4e97694a0c1182173e9b569
I20260619 12:09:46.712409 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42459
--builtin_ntp_servers=127.0.201.20:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:09:46.791853 5631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:46.792042 5631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:46.792061 5631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:46.793118 5631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:46.793152 5631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:46.793177 5631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:46.794431 5631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:46.794626 5631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:46.794858 5631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:46.799795 5636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.799782 5637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.799888 5631 server_base.cc:1034] running on GCE node
W20260619 12:09:46.799780 5639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.800240 5631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:46.800428 5631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:46.801596 5631 hybrid_clock.cc:648] HybridClock initialized: now 1781870986801575 us; error 37 us; skew 500 ppm
I20260619 12:09:46.802758 5631 webserver.cc:460] Webserver started at http://127.0.201.2:42547/ using document root <none> and password file <none>
I20260619 12:09:46.802985 5631 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:46.803030 5631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:46.803184 5631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:46.804229 5631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "fa307f2eb6614cddb12e9b40d4ef9869"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "4a4f55a4f35b38aa3dfd5739458d1e49"
server_key_iv: "c389c4b740101555952086361715e50c"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.804571 5631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "fa307f2eb6614cddb12e9b40d4ef9869"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "4a4f55a4f35b38aa3dfd5739458d1e49"
server_key_iv: "c389c4b740101555952086361715e50c"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.805830 5631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:09:46.806468 5645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.806687 5631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:46.806777 5631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "fa307f2eb6614cddb12e9b40d4ef9869"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "4a4f55a4f35b38aa3dfd5739458d1e49"
server_key_iv: "c389c4b740101555952086361715e50c"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.806851 5631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.810220 5631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.810467 5631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.810604 5631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.810791 5631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:46.811089 5631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:46.811146 5631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.811183 5631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:46.811228 5631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.816807 5631 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:46225
I20260619 12:09:46.816836 5758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:46225 every 8 connection(s)
I20260619 12:09:46.817152 5631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:46.820116 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.817899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:41859 (local address 127.0.201.62:42459)
0619 12:09:46.817969 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.817971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.818166 (+ 195us) server_negotiation.cc:407] Connection header received
0619 12:09:46.818648 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.818649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.818654 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.818701 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.819904 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.819906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.819906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.819910 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.819924 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.819928 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.819975 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.820037 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.820048 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:09:46.820299 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.817851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42459 (local address 127.0.201.2:41859)
0619 12:09:46.818039 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.818073 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.818561 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.818764 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.818767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.818775 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.819763 (+ 988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:46.819763 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.819795 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:46.819824 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.819976 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.819978 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.819980 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.820036 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.820087 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260619 12:09:46.820775 5759 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42459
I20260619 12:09:46.820864 5759 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:46.820982 5759 heartbeater.cc:511] Master 127.0.201.62:42459 requested a full tablet report, sending...
I20260619 12:09:46.821336 5440 ts_manager.cc:194] Registered new tserver with Master: fa307f2eb6614cddb12e9b40d4ef9869 (127.0.201.2:46225)
I20260619 12:09:46.827086 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5631
I20260619 12:09:46.827170 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:46.827412 804 external_mini_cluster.cc:1428] Setting key 60657f8ed971128017d77d136fa73463
I20260619 12:09:46.828361 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42459
--builtin_ntp_servers=127.0.201.20:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:09:46.910492 5762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:46.910728 5762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:46.910763 5762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:46.911985 5762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:46.912057 5762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:46.912101 5762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:46.913463 5762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:46.913743 5762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:46.914041 5762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:46.919083 5767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.919117 5768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:46.919116 5770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:46.919350 5762 server_base.cc:1034] running on GCE node
I20260619 12:09:46.919592 5762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:46.919816 5762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:46.920984 5762 hybrid_clock.cc:648] HybridClock initialized: now 1781870986920958 us; error 39 us; skew 500 ppm
I20260619 12:09:46.922330 5762 webserver.cc:460] Webserver started at http://127.0.201.3:44733/ using document root <none> and password file <none>
I20260619 12:09:46.922622 5762 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:46.922680 5762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:46.922868 5762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:46.924141 5762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "5f75c01924204a66b80ffee9cbff125a"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "dad22d03d00b194293801c71d8758ba2"
server_key_iv: "bfbff10c04d45f7479a5bbddfbd8c15e"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.924546 5762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "5f75c01924204a66b80ffee9cbff125a"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "dad22d03d00b194293801c71d8758ba2"
server_key_iv: "bfbff10c04d45f7479a5bbddfbd8c15e"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.925907 5762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:09:46.926640 5776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.926806 5762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:46.926915 5762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "5f75c01924204a66b80ffee9cbff125a"
format_stamp: "Formatted at 2026-06-19 12:09:46 on dist-test-slave-c910"
server_key: "dad22d03d00b194293801c71d8758ba2"
server_key_iv: "bfbff10c04d45f7479a5bbddfbd8c15e"
server_key_version: "encryptionkey@0"
I20260619 12:09:46.927006 5762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:46.930754 5762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:46.931054 5762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:46.931264 5762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:46.931507 5762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:46.931843 5762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:46.931901 5762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.931936 5762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:46.931959 5762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:46.938324 5762 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:43895
I20260619 12:09:46.938362 5889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:43895 every 8 connection(s)
I20260619 12:09:46.938757 5762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:46.941744 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.939559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:43597 (local address 127.0.201.62:42459)
0619 12:09:46.939644 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.939646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.939839 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:09:46.940340 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.940341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.940345 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.940393 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.941548 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.941550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.941550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.941554 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.941569 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.941574 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.941619 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.941681 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.941692 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:09:46.942004 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.939507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42459 (local address 127.0.201.3:43597)
0619 12:09:46.939703 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.939740 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.940240 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.940464 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.940468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.940476 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.941382 (+ 906us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:46.941383 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.941421 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:46.941450 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.941619 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.941621 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.941624 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.941680 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.941732 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260619 12:09:46.942553 5890 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42459
I20260619 12:09:46.942636 5890 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:46.942828 5890 heartbeater.cc:511] Master 127.0.201.62:42459 requested a full tablet report, sending...
I20260619 12:09:46.942940 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5762
I20260619 12:09:46.943064 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:46.943279 5440 ts_manager.cc:194] Registered new tserver with Master: 5f75c01924204a66b80ffee9cbff125a (127.0.201.3:43895)
I20260619 12:09:46.943307 804 external_mini_cluster.cc:1428] Setting key f0f80729fa213368b9aa365bf25fa188
I20260619 12:09:46.944029 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:09:46.946688 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.945520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.0.201.62:42459)
0619 12:09:46.945591 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.945593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.945672 (+ 79us) server_negotiation.cc:407] Connection header received
0619 12:09:46.945759 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.945760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.945766 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.945829 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.946506 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.946507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.946507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.946511 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.946525 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.946528 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.946572 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.946625 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.946636 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:09:46.948215 5440 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:46.949955 5440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:46.954066 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.952229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:43895 (local address 127.0.0.1:40552)
0619 12:09:46.952557 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.952569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.952634 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.953172 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.953172 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.953175 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.953592 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:46.953594 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.953598 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:46.953609 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.953846 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.953847 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.953848 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.953957 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.953970 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":93,"threads_started":1}
I20260619 12:09:46.954149 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.952542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.0.201.1:40727)
0619 12:09:46.952722 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.952727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.952735 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:46.952760 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.952763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.952772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.952819 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.953645 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.953653 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.953653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.953658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.953697 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.953726 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.953805 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.954017 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.954035 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260619 12:09:46.954236 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.952295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.0.201.2:46225)
0619 12:09:46.952452 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.952457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.952615 (+ 158us) server_negotiation.cc:407] Connection header received
0619 12:09:46.952726 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.952730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.952741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.952795 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.953900 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.953908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.953908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.953912 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.953944 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.953968 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.954027 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.954125 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.954141 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260619 12:09:46.954372 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.952162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:46225 (local address 127.0.0.1:43416)
0619 12:09:46.952488 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.952530 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.952643 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.953173 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.953174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.953176 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.953803 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:46.953804 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.953809 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:46.953819 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.954053 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.954053 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.954054 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.954306 (+ 252us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.954320 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":69,"threads_started":1}
I20260619 12:09:46.954403 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.952308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.0.201.3:43895)
0619 12:09:46.952534 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:46.952538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:46.952697 (+ 159us) server_negotiation.cc:407] Connection header received
0619 12:09:46.952724 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:46.952727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:46.952737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:46.952804 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:46.953705 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:46.953711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:46.953712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:46.953716 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:46.953745 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:46.953768 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:46.953823 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:46.954302 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 12:09:46.954321 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260619 12:09:46.953976 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:46.952369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:40727 (local address 127.0.0.1:35878)
0619 12:09:46.952583 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:46.952592 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:46.952634 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:46.953004 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:46.953009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:46.953017 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:46.953493 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:46.953494 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:46.953528 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:46.953553 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:46.953805 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:46.953812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:46.953815 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:46.953886 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:09:46.953900 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":62,"threads_started":1}
I20260619 12:09:46.955051 5693 tablet_service.cc:1467] Processing CreateTablet for tablet 31452d15b09043429e5047864c81fa6f (DEFAULT_TABLE table=test-table [id=8bce28a825944d578276260c7113f34b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:46.955051 5824 tablet_service.cc:1467] Processing CreateTablet for tablet 31452d15b09043429e5047864c81fa6f (DEFAULT_TABLE table=test-table [id=8bce28a825944d578276260c7113f34b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:46.955428 5693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31452d15b09043429e5047864c81fa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.955428 5824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31452d15b09043429e5047864c81fa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.955770 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 31452d15b09043429e5047864c81fa6f (DEFAULT_TABLE table=test-table [id=8bce28a825944d578276260c7113f34b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:46.956027 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31452d15b09043429e5047864c81fa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:46.958443 5904 tablet_bootstrap.cc:492] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d: Bootstrap starting.
I20260619 12:09:46.958833 5906 tablet_bootstrap.cc:492] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Bootstrap starting.
I20260619 12:09:46.959105 5904 tablet_bootstrap.cc:654] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.959353 5905 tablet_bootstrap.cc:492] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Bootstrap starting.
I20260619 12:09:46.959410 5906 tablet_bootstrap.cc:654] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.959481 5904 log.cc:824] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.959719 5906 log.cc:824] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.959995 5905 tablet_bootstrap.cc:654] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:46.960285 5904 tablet_bootstrap.cc:492] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d: No bootstrap required, opened a new log
I20260619 12:09:46.960292 5905 log.cc:824] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:46.960358 5904 ts_tablet_manager.cc:1397] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:46.960466 5906 tablet_bootstrap.cc:492] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: No bootstrap required, opened a new log
I20260619 12:09:46.960527 5906 ts_tablet_manager.cc:1397] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:46.960968 5905 tablet_bootstrap.cc:492] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: No bootstrap required, opened a new log
I20260619 12:09:46.961047 5905 ts_tablet_manager.cc:1397] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:46.962782 5905 raft_consensus.cc:348] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:46.962934 5905 raft_consensus.cc:374] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.962977 5905 raft_consensus.cc:729] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa307f2eb6614cddb12e9b40d4ef9869, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.963006 5906 raft_consensus.cc:348] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:46.962986 5904 raft_consensus.cc:348] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:46.963147 5906 raft_consensus.cc:374] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.963148 5904 raft_consensus.cc:374] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:46.963111 5905 consensus_queue.cc:260] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:46.963183 5904 raft_consensus.cc:729] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3918f4be04bd4773926a3a84e6ddce7d, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.963183 5906 raft_consensus.cc:729] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f75c01924204a66b80ffee9cbff125a, State: Initialized, Role: FOLLOWER
I20260619 12:09:46.963316 5906 consensus_queue.cc:260] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:46.963361 5904 consensus_queue.cc:260] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:46.963430 5905 ts_tablet_manager.cc:1428] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:09:46.963550 5759 heartbeater.cc:503] Master 127.0.201.62:42459 was elected leader, sending a full tablet report...
I20260619 12:09:46.963572 5904 ts_tablet_manager.cc:1428] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 12:09:46.963574 5906 ts_tablet_manager.cc:1428] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 12:09:46.963685 5890 heartbeater.cc:503] Master 127.0.201.62:42459 was elected leader, sending a full tablet report...
I20260619 12:09:46.963948 5628 heartbeater.cc:503] Master 127.0.201.62:42459 was elected leader, sending a full tablet report...
W20260619 12:09:47.067735 5760 tablet.cc:2369] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:47.068857 5912 raft_consensus.cc:482] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:47.069064 5912 raft_consensus.cc:504] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:47.069422 5912 leader_election.cc:288] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa307f2eb6614cddb12e9b40d4ef9869 (127.0.201.2:46225), 5f75c01924204a66b80ffee9cbff125a (127.0.201.3:43895)
I20260619 12:09:47.070822 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.069616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:46225 (local address 127.0.201.1:34873)
0619 12:09:47.069697 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:47.069711 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:47.069763 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:47.069926 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:47.069927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:47.069929 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:47.070333 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:47.070333 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:47.070339 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:47.070351 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:47.070649 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:47.070650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:47.070651 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:47.070725 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:09:47.070741 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 12:09:47.070866 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.069714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:34873 (local address 127.0.201.2:46225)
0619 12:09:47.069760 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.069762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.069773 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:47.069842 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.069843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.069847 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.069896 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.070522 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.070525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.070525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.070530 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.070552 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.070557 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.070622 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.070760 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.070776 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:47.071069 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.069683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47293 (local address 127.0.201.3:43895)
0619 12:09:47.069728 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.069730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.069946 (+ 216us) server_negotiation.cc:407] Connection header received
0619 12:09:47.069978 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.069979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.069984 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.070026 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.070767 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.070769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.070769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.070773 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.070790 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.070794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.070855 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.070958 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.070975 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:47.071303 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.069616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:43895 (local address 127.0.201.1:47293)
0619 12:09:47.069843 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:47.069852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:47.069924 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:47.070046 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:47.070047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:47.070060 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:47.070646 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:09:47.070647 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:47.070653 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:09:47.070671 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:47.070866 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:47.070867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:47.070867 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 12:09:47.071216 (+ 349us) client_negotiation.cc:240] Negotiation successful
0619 12:09:47.071230 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":62,"thread_start_us":90,"threads_started":1}
I20260619 12:09:47.071266 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31452d15b09043429e5047864c81fa6f" candidate_uuid: "3918f4be04bd4773926a3a84e6ddce7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" is_pre_election: true
I20260619 12:09:47.071414 5713 raft_consensus.cc:2454] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3918f4be04bd4773926a3a84e6ddce7d in term 0.
I20260619 12:09:47.071650 5844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31452d15b09043429e5047864c81fa6f" candidate_uuid: "3918f4be04bd4773926a3a84e6ddce7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f75c01924204a66b80ffee9cbff125a" is_pre_election: true
I20260619 12:09:47.071801 5844 raft_consensus.cc:2454] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3918f4be04bd4773926a3a84e6ddce7d in term 0.
I20260619 12:09:47.072036 5518 leader_election.cc:302] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3918f4be04bd4773926a3a84e6ddce7d, fa307f2eb6614cddb12e9b40d4ef9869; no voters:
I20260619 12:09:47.072182 5912 raft_consensus.cc:2781] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:47.072256 5912 raft_consensus.cc:482] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:47.072291 5912 raft_consensus.cc:3037] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.072997 5912 raft_consensus.cc:504] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:47.073145 5912 leader_election.cc:288] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [CANDIDATE]: Term 1 election: Requested vote from peers fa307f2eb6614cddb12e9b40d4ef9869 (127.0.201.2:46225), 5f75c01924204a66b80ffee9cbff125a (127.0.201.3:43895)
I20260619 12:09:47.073297 5844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31452d15b09043429e5047864c81fa6f" candidate_uuid: "3918f4be04bd4773926a3a84e6ddce7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f75c01924204a66b80ffee9cbff125a"
I20260619 12:09:47.073318 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31452d15b09043429e5047864c81fa6f" candidate_uuid: "3918f4be04bd4773926a3a84e6ddce7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa307f2eb6614cddb12e9b40d4ef9869"
I20260619 12:09:47.073390 5713 raft_consensus.cc:3037] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.073390 5844 raft_consensus.cc:3037] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.074275 5713 raft_consensus.cc:2454] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3918f4be04bd4773926a3a84e6ddce7d in term 1.
I20260619 12:09:47.074275 5844 raft_consensus.cc:2454] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3918f4be04bd4773926a3a84e6ddce7d in term 1.
I20260619 12:09:47.074429 5518 leader_election.cc:302] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3918f4be04bd4773926a3a84e6ddce7d, fa307f2eb6614cddb12e9b40d4ef9869; no voters:
I20260619 12:09:47.074544 5912 raft_consensus.cc:2781] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:47.074714 5912 raft_consensus.cc:686] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [term 1 LEADER]: Becoming Leader. State: Replica: 3918f4be04bd4773926a3a84e6ddce7d, State: Running, Role: LEADER
I20260619 12:09:47.074808 5912 consensus_queue.cc:237] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } }
I20260619 12:09:47.075573 5439 catalog_manager.cc:5495] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3918f4be04bd4773926a3a84e6ddce7d (127.0.201.1). New cstate: current_term: 1 leader_uuid: "3918f4be04bd4773926a3a84e6ddce7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3918f4be04bd4773926a3a84e6ddce7d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 40727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:47.099972 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.098962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.0.201.1:40727)
0619 12:09:47.099143 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.099145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.099151 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:09:47.099183 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.099184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.099234 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.099296 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.099781 (+ 485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.099782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.099782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.099785 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.099799 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.099802 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.099851 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.099884 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.099895 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260619 12:09:47.101752 5844 raft_consensus.cc:1261] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 1 FOLLOWER]: Refusing update from remote peer 3918f4be04bd4773926a3a84e6ddce7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:47.101786 5713 raft_consensus.cc:1261] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 1 FOLLOWER]: Refusing update from remote peer 3918f4be04bd4773926a3a84e6ddce7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:47.102093 5914 consensus_queue.cc:1035] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f75c01924204a66b80ffee9cbff125a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:47.102229 5912 consensus_queue.cc:1035] T 31452d15b09043429e5047864c81fa6f P 3918f4be04bd4773926a3a84e6ddce7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa307f2eb6614cddb12e9b40d4ef9869" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 46225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:47.104701 5918 mvcc.cc:205] Tried to move back new op lower bound from 7298543563166113792 to 7298543563059019776. Current Snapshot: MvccSnapshot[applied={T|T < 7298543563166113792}]
I20260619 12:09:47.107133 5439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:47.107251 5439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:47.107988 5439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a32a9c8ffc944349b3615a7aee3a657: Sending DeleteTablet for 3 replicas of tablet 31452d15b09043429e5047864c81fa6f
I20260619 12:09:47.108327 5824 tablet_service.cc:1514] Processing DeleteTablet for tablet 31452d15b09043429e5047864c81fa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:47 UTC) from {username='slave'} at 127.0.0.1:40552
I20260619 12:09:47.108361 5693 tablet_service.cc:1514] Processing DeleteTablet for tablet 31452d15b09043429e5047864c81fa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:47 UTC) from {username='slave'} at 127.0.0.1:43416
I20260619 12:09:47.108564 5927 tablet_replica.cc:330] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: stopping tablet replica
I20260619 12:09:47.108685 5927 raft_consensus.cc:2229] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:47.108716 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet 31452d15b09043429e5047864c81fa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:47 UTC) from {username='slave'} at 127.0.0.1:35878
I20260619 12:09:47.108793 5927 raft_consensus.cc:2258] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:47.108800 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5500
I20260619 12:09:47.108946 5928 tablet_replica.cc:330] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: stopping tablet replica
I20260619 12:09:47.109030 5928 raft_consensus.cc:2229] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:47.109117 5928 raft_consensus.cc:2258] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:47.109894 5927 ts_tablet_manager.cc:1905] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:47.111932 5927 ts_tablet_manager.cc:1918] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:47.112011 5927 log.cc:1196] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/31452d15b09043429e5047864c81fa6f
I20260619 12:09:47.112288 5927 ts_tablet_manager.cc:1939] T 31452d15b09043429e5047864c81fa6f P fa307f2eb6614cddb12e9b40d4ef9869: Deleting consensus metadata
I20260619 12:09:47.112717 5427 catalog_manager.cc:4841] TS fa307f2eb6614cddb12e9b40d4ef9869 (127.0.201.2:46225): tablet 31452d15b09043429e5047864c81fa6f (table test-table [id=8bce28a825944d578276260c7113f34b]) successfully deleted
I20260619 12:09:47.113039 5928 ts_tablet_manager.cc:1905] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:47.114928 5928 ts_tablet_manager.cc:1918] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:47.115003 5928 log.cc:1196] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/31452d15b09043429e5047864c81fa6f
I20260619 12:09:47.115267 5928 ts_tablet_manager.cc:1939] T 31452d15b09043429e5047864c81fa6f P 5f75c01924204a66b80ffee9cbff125a: Deleting consensus metadata
I20260619 12:09:47.115687 5424 catalog_manager.cc:4841] TS 5f75c01924204a66b80ffee9cbff125a (127.0.201.3:43895): tablet 31452d15b09043429e5047864c81fa6f (table test-table [id=8bce28a825944d578276260c7113f34b]) successfully deleted
W20260619 12:09:47.116379 5427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:40727 (error 108)
I20260619 12:09:47.116586 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5631
I20260619 12:09:47.117187 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.116689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:40727 (local address 127.0.0.1:35894)
0619 12:09:47.117107 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:47.117147 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397}
W20260619 12:09:47.117352 5427 catalog_manager.cc:4584] TS 3918f4be04bd4773926a3a84e6ddce7d (127.0.201.1:40727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31452d15b09043429e5047864c81fa6f: Network error: Client connection negotiation failed: client connection to 127.0.201.1:40727: connect: Connection refused (error 111)
I20260619 12:09:47.121932 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5762
I20260619 12:09:47.126220 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5409
2026-06-19T12:09:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (717 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 12:09:47.145437 804 mini_oidc.cc:152] Starting JWKS server
I20260619 12:09:47.146513 804 webserver.cc:460] Webserver started at https://127.0.0.1:38707/ using document root <none> and password file <none>
I20260619 12:09:47.146600 804 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:09:47.146796 804 webserver.cc:460] Webserver started at http://127.0.0.1:43975/ using document root <none> and password file <none>
2026-06-19T12:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:47Z Disabled control of system clock
I20260619 12:09:47.154309 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:43547
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:45311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:43547
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38707/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/testchainca.cert with env {}
W20260619 12:09:47.232633 5936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:47.232841 5936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:47.232863 5936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:47.234016 5936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:47.234055 5936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:47.234068 5936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:47.234081 5936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:47.234133 5936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:09:47.234150 5936 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:09:47.234171 5936 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38707/jwks.json
I20260619 12:09:47.235764 5936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:45311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:43547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:43547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38707/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:47.236019 5936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:47.236243 5936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:47.240856 5942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:47.240845 5941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:47.240846 5944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:47.241267 5936 server_base.cc:1034] running on GCE node
I20260619 12:09:47.241531 5936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:47.241811 5936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:47.243031 5936 hybrid_clock.cc:648] HybridClock initialized: now 1781870987243009 us; error 31 us; skew 500 ppm
I20260619 12:09:47.244300 5936 webserver.cc:460] Webserver started at http://127.0.201.62:32977/ using document root <none> and password file <none>
I20260619 12:09:47.244558 5936 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:47.244630 5936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:47.244779 5936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:47.245843 5936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "244b52a5267a4b1683c2abaf1dee0d80"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "b6cd930690dfddbf9e3764ab5749c895"
server_key_iv: "c06ede552251ebbb67222765572de700"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.246203 5936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "244b52a5267a4b1683c2abaf1dee0d80"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "b6cd930690dfddbf9e3764ab5749c895"
server_key_iv: "c06ede552251ebbb67222765572de700"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.247637 5936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:47.248422 5950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:47.248593 5936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:47.248692 5936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "244b52a5267a4b1683c2abaf1dee0d80"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "b6cd930690dfddbf9e3764ab5749c895"
server_key_iv: "c06ede552251ebbb67222765572de700"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.248767 5936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:47.282271 5936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:47.282645 5936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:47.282817 5936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:47.286602 5936 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:43547
I20260619 12:09:47.286634 6002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:43547 every 8 connection(s)
I20260619 12:09:47.286988 5936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:47.287964 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:47.288537 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 5936
I20260619 12:09:47.288653 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:47.288825 804 external_mini_cluster.cc:1428] Setting key 9ce7b92cbaf5f795b41d4e817d63e2bf
I20260619 12:09:47.290973 6003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80: Bootstrap starting.
I20260619 12:09:47.291692 6003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:47.291981 6003 log.cc:824] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:47.292660 6003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80: No bootstrap required, opened a new log
I20260619 12:09:47.294622 6003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244b52a5267a4b1683c2abaf1dee0d80" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43547 } }
I20260619 12:09:47.294771 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.289357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.0.201.62:43547)
0619 12:09:47.289578 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.289587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.289620 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:09:47.290150 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.290159 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.290244 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.290399 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.290583 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.291915 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.292792 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.293230 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.293387 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.294090 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.294094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.294094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.294261 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.294294 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.294304 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.294376 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.294491 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.294572 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260619 12:09:47.294834 6003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:47.294914 6003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 244b52a5267a4b1683c2abaf1dee0d80, State: Initialized, Role: FOLLOWER
I20260619 12:09:47.295045 6003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244b52a5267a4b1683c2abaf1dee0d80" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43547 } }
I20260619 12:09:47.295149 6003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:47.295213 6003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:47.295269 6003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.295818 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:47.295957 6003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244b52a5267a4b1683c2abaf1dee0d80" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43547 } }
I20260619 12:09:47.296075 6003 leader_election.cc:302] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 244b52a5267a4b1683c2abaf1dee0d80; no voters:
I20260619 12:09:47.296248 6003 leader_election.cc:288] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:47.296291 6008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:47.296407 6008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [term 1 LEADER]: Becoming Leader. State: Replica: 244b52a5267a4b1683c2abaf1dee0d80, State: Running, Role: LEADER
I20260619 12:09:47.296485 6008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244b52a5267a4b1683c2abaf1dee0d80" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43547 } }
I20260619 12:09:47.296514 6003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:47.296836 6008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 244b52a5267a4b1683c2abaf1dee0d80. Latest consensus state: current_term: 1 leader_uuid: "244b52a5267a4b1683c2abaf1dee0d80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244b52a5267a4b1683c2abaf1dee0d80" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43547 } } }
I20260619 12:09:47.296911 6008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.296859 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "244b52a5267a4b1683c2abaf1dee0d80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244b52a5267a4b1683c2abaf1dee0d80" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43547 } } }
I20260619 12:09:47.296967 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.297683 6014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:47.298200 6014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:47.299661 6014 catalog_manager.cc:1347] Generated new cluster ID: 5c766ee6b16b4d5f9bfaed35c3329fda
I20260619 12:09:47.299712 6014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:47.313287 6014 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:47.313988 6014 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:47.331218 6014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 244b52a5267a4b1683c2abaf1dee0d80: Generated new TSK 0
I20260619 12:09:47.331475 6014 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 12:09:47.360523 6033 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 12:09:47.360635 6033 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 12:09:47.360853 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.345181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.0.201.62:43547)
0619 12:09:47.345235 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.345236 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.345351 (+ 115us) server_negotiation.cc:407] Connection header received
0619 12:09:47.345478 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.345480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.345491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.345558 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:09:47.345746 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.346314 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.347508 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.347701 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.347761 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.347981 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:09:47.360613 (+ 12632us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 12:09:47.360785 (+ 172us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42614: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 12:09:47.360924 6006 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42614: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 12:09:47.361228 6032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:47.345058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:43547 (local address 127.0.0.1:42614)
0619 12:09:47.345270 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:47.345282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:47.345358 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:47.345594 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:47.345595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:47.345601 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:09:47.345675 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:47.345678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.346407 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.346408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:47.347411 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:47.347414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.347779 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.347780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:47.347846 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.347867 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:09:47.360832 (+ 12965us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 12:09:47.360946 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.201.62:43547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.201.62:43547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 12:09:47.361672 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 5936
2026-06-19T12:09:47Z chronyd exiting
I20260619 12:09:47.397318 804 test_util.cc:175] -----------------------------------------------
I20260619 12:09:47.397396 804 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781870963166527-804-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (252 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 12:09:47.398190 804 mini_oidc.cc:152] Starting JWKS server
I20260619 12:09:47.398886 804 webserver.cc:460] Webserver started at https://127.0.0.1:35565/ using document root <none> and password file <none>
I20260619 12:09:47.398967 804 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:09:47.399151 804 webserver.cc:460] Webserver started at http://127.0.0.1:38887/ using document root <none> and password file <none>
2026-06-19T12:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:47Z Disabled control of system clock
I20260619 12:09:47.406635 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44269
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:44269
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35565/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/testchainca.cert with env {}
W20260619 12:09:47.486259 6039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:47.486469 6039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:47.486490 6039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:47.487679 6039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:47.487722 6039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:47.487735 6039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:47.487749 6039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:47.487761 6039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:09:47.487819 6039 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:09:47.487836 6039 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35565/jwks.json
I20260619 12:09:47.489360 6039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:44269
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35565/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:47.489619 6039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:47.489845 6039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:47.494887 6047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:47.494935 6039 server_base.cc:1034] running on GCE node
W20260619 12:09:47.494825 6045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:47.494822 6044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:47.495468 6039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:47.495833 6039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:47.497041 6039 hybrid_clock.cc:648] HybridClock initialized: now 1781870987497026 us; error 31 us; skew 500 ppm
I20260619 12:09:47.498521 6039 webserver.cc:460] Webserver started at http://127.0.201.62:41479/ using document root <none> and password file <none>
I20260619 12:09:47.498787 6039 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:47.498862 6039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:47.499018 6039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:47.500170 6039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "a2a0e635549c4fb09d44613f140f60cf"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "8759970f9fcfdfc2190871a7830bf480"
server_key_iv: "40297b2dd359f3defdce31628295fda2"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.500551 6039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "a2a0e635549c4fb09d44613f140f60cf"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "8759970f9fcfdfc2190871a7830bf480"
server_key_iv: "40297b2dd359f3defdce31628295fda2"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.501848 6039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:47.502792 6053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:47.503015 6039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:47.503118 6039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "a2a0e635549c4fb09d44613f140f60cf"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "8759970f9fcfdfc2190871a7830bf480"
server_key_iv: "40297b2dd359f3defdce31628295fda2"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.503225 6039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:47.520758 6039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:47.521140 6039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:47.521308 6039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:47.525362 6039 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:44269
I20260619 12:09:47.525403 6105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:44269 every 8 connection(s)
I20260619 12:09:47.525754 6039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:47.526721 6106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:47.529034 6106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf: Bootstrap starting.
I20260619 12:09:47.529644 6106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:47.529927 6106 log.cc:824] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:47.530620 6106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf: No bootstrap required, opened a new log
I20260619 12:09:47.531054 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6039
I20260619 12:09:47.531149 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:47.531355 804 external_mini_cluster.cc:1428] Setting key ad73bd25b5e5f5e833225b8da921deaa
I20260619 12:09:47.533084 6106 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a0e635549c4fb09d44613f140f60cf" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44269 } }
I20260619 12:09:47.533221 6106 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:47.533245 6106 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a0e635549c4fb09d44613f140f60cf, State: Initialized, Role: FOLLOWER
I20260619 12:09:47.533372 6106 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a0e635549c4fb09d44613f140f60cf" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44269 } }
I20260619 12:09:47.533442 6106 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:47.533463 6106 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:47.533533 6106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.534104 6106 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a0e635549c4fb09d44613f140f60cf" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44269 } }
I20260619 12:09:47.534190 6106 leader_election.cc:302] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2a0e635549c4fb09d44613f140f60cf; no voters:
I20260619 12:09:47.534372 6106 leader_election.cc:288] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:47.534394 6111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:47.534498 6111 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [term 1 LEADER]: Becoming Leader. State: Replica: a2a0e635549c4fb09d44613f140f60cf, State: Running, Role: LEADER
I20260619 12:09:47.534584 6111 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a0e635549c4fb09d44613f140f60cf" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44269 } }
I20260619 12:09:47.534670 6106 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:47.534896 6111 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2a0e635549c4fb09d44613f140f60cf. Latest consensus state: current_term: 1 leader_uuid: "a2a0e635549c4fb09d44613f140f60cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a0e635549c4fb09d44613f140f60cf" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44269 } } }
I20260619 12:09:47.534904 6112 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2a0e635549c4fb09d44613f140f60cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a0e635549c4fb09d44613f140f60cf" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44269 } } }
I20260619 12:09:47.534966 6111 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.535000 6112 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.535576 6116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:47.536295 6116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:47.537904 6116 catalog_manager.cc:1347] Generated new cluster ID: 57e485a4f5354cc1b9629a2f7e7373e9
I20260619 12:09:47.537954 6116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:47.538326 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.531887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.0.201.62:44269)
0619 12:09:47.532090 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.532098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.532123 (+ 25us) server_negotiation.cc:407] Connection header received
0619 12:09:47.532567 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.532574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.532632 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.532749 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.533005 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.534447 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.535616 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.536257 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.536374 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.537467 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.537470 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.537471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.537701 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.537743 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.537757 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.537820 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.538105 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.538169 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260619 12:09:47.539284 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:47.545624 6116 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:47.546172 6116 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:47.549309 6116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2a0e635549c4fb09d44613f140f60cf: Generated new TSK 0
I20260619 12:09:47.549562 6116 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 12:09:47.573299 6136 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 12:09:47.573437 6136 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 12:09:47.573725 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.558417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.0.201.62:44269)
0619 12:09:47.558479 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.558481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.558562 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:09:47.558617 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.558618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.558629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.558715 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:09:47.558800 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.559628 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.560444 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.560764 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.560835 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.561023 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:09:47.573485 (+ 12462us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 12:09:47.573653 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43532: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 12:09:47.573786 6110 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43532: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 12:09:47.573807 6135 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:47.558336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44269 (local address 127.0.0.1:43532)
0619 12:09:47.558489 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:47.558499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:47.558563 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:47.558711 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:47.558711 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:47.558713 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:09:47.558763 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:47.558765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.559749 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.559751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:47.560356 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:47.560358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.560840 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.560841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:47.560921 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.560926 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:09:47.573618 (+ 12692us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 12:09:47.573712 (+ 94us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.201.62:44269: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
I20260619 12:09:47.574156 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6039
2026-06-19T12:09:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 12:09:47.611788 804 mini_oidc.cc:152] Starting JWKS server
I20260619 12:09:47.612550 804 webserver.cc:460] Webserver started at https://127.0.0.1:33389/ using document root <none> and password file <none>
I20260619 12:09:47.612628 804 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:09:47.612815 804 webserver.cc:460] Webserver started at http://127.0.0.1:38969/ using document root <none> and password file <none>
2026-06-19T12:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:47Z Disabled control of system clock
I20260619 12:09:47.620798 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:43981
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:43981
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33389/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 12:09:47.703416 6142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:47.703724 6142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:47.703761 6142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:47.705382 6142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:47.705430 6142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:47.705446 6142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:47.705461 6142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:47.705497 6142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:09:47.705530 6142 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:09:47.705552 6142 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33389/jwks.json
I20260619 12:09:47.706941 6142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:43981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:43981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33389/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:47.707275 6142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:47.707620 6142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:47.712606 6147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:47.712667 6142 server_base.cc:1034] running on GCE node
W20260619 12:09:47.712589 6150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:47.712638 6148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:47.713114 6142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:47.713419 6142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:47.714578 6142 hybrid_clock.cc:648] HybridClock initialized: now 1781870987714567 us; error 27 us; skew 500 ppm
I20260619 12:09:47.715845 6142 webserver.cc:460] Webserver started at http://127.0.201.62:44899/ using document root <none> and password file <none>
I20260619 12:09:47.716056 6142 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:47.716101 6142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:47.716194 6142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:47.717231 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "5c0aa8e782484459b26c62f1a888106d"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "d1b3974d09d00512104d7e562d13ad1d"
server_key_iv: "32c2a4efc57930b5ded50bb250d6abb6"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.717602 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "5c0aa8e782484459b26c62f1a888106d"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "d1b3974d09d00512104d7e562d13ad1d"
server_key_iv: "32c2a4efc57930b5ded50bb250d6abb6"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.718816 6142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:47.719547 6156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:47.719699 6142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:47.719786 6142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "5c0aa8e782484459b26c62f1a888106d"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "d1b3974d09d00512104d7e562d13ad1d"
server_key_iv: "32c2a4efc57930b5ded50bb250d6abb6"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.719854 6142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:47.754216 6142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:47.754522 6142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:47.754647 6142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:47.758519 6142 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:43981
I20260619 12:09:47.758600 6208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:43981 every 8 connection(s)
I20260619 12:09:47.758929 6142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:47.759668 6209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:47.762324 6209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d: Bootstrap starting.
I20260619 12:09:47.762934 6209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:47.763257 6209 log.cc:824] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:47.763985 6209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d: No bootstrap required, opened a new log
I20260619 12:09:47.765331 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6142
I20260619 12:09:47.765440 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:47.765605 804 external_mini_cluster.cc:1428] Setting key fb99bd6723fa2f383a67547c07398737
I20260619 12:09:47.765880 6209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0aa8e782484459b26c62f1a888106d" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43981 } }
I20260619 12:09:47.766076 6209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:47.766127 6209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0aa8e782484459b26c62f1a888106d, State: Initialized, Role: FOLLOWER
I20260619 12:09:47.766238 6209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0aa8e782484459b26c62f1a888106d" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43981 } }
I20260619 12:09:47.766353 6209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:47.766400 6209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:47.766474 6209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.767010 6209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0aa8e782484459b26c62f1a888106d" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43981 } }
I20260619 12:09:47.767127 6209 leader_election.cc:302] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c0aa8e782484459b26c62f1a888106d; no voters:
I20260619 12:09:47.767346 6209 leader_election.cc:288] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:47.767366 6214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:47.767482 6214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [term 1 LEADER]: Becoming Leader. State: Replica: 5c0aa8e782484459b26c62f1a888106d, State: Running, Role: LEADER
I20260619 12:09:47.767563 6214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0aa8e782484459b26c62f1a888106d" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43981 } }
I20260619 12:09:47.767601 6209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:47.767904 6215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c0aa8e782484459b26c62f1a888106d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0aa8e782484459b26c62f1a888106d" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43981 } } }
I20260619 12:09:47.767983 6215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.768126 6216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c0aa8e782484459b26c62f1a888106d. Latest consensus state: current_term: 1 leader_uuid: "5c0aa8e782484459b26c62f1a888106d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c0aa8e782484459b26c62f1a888106d" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43981 } } }
I20260619 12:09:47.768177 6216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.768253 6218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:47.768926 6218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:47.770852 6218 catalog_manager.cc:1347] Generated new cluster ID: 398266244ead42cf9851259d950487d8
I20260619 12:09:47.770921 6218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:47.772078 6213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.766129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.0.201.62:43981)
0619 12:09:47.766329 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.766337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.766387 (+ 50us) server_negotiation.cc:407] Connection header received
0619 12:09:47.766498 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.766504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.766602 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.766744 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.766921 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.768620 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.769752 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.770333 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.770441 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.771330 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.771334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.771334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.771513 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.771554 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.771566 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.771614 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.771817 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.771885 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260619 12:09:47.773044 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:47.792882 6218 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:47.793367 6218 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:47.806906 6218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c0aa8e782484459b26c62f1a888106d: Generated new TSK 0
I20260619 12:09:47.807130 6218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:47.823087 6213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.819463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.0.201.62:43981)
0619 12:09:47.819510 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.819512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.819589 (+ 77us) server_negotiation.cc:407] Connection header received
0619 12:09:47.819671 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.819671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.819682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.819749 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:09:47.819913 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.820381 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.821493 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.821664 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.821711 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.821933 (+ 222us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:09:47.822873 (+ 940us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33389/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 12:09:47.822996 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33389/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 12:09:47.823151 6213 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33389/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 12:09:47.823453 6238 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:47.819354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:43981 (local address 127.0.0.1:59272)
0619 12:09:47.819511 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:47.819524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:47.819595 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:47.819768 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:47.819768 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:47.819771 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:09:47.819842 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:47.819844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.820478 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.820479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:47.821407 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:47.821411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.821768 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.821769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:47.821837 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.821843 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:09:47.822967 (+ 1124us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33389/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 12:09:47.823077 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.201.62:43981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33389/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260619 12:09:47.823700 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6142
E20260619 12:09:47.833906 6239 webserver.cc:533] Webserver: sslize failed:
E20260619 12:09:47.833983 6239 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T12:09:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T12:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:47Z Disabled control of system clock
I20260619 12:09:47.867841 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:41003
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:41003
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/testchainca.cert with env {}
W20260619 12:09:47.946699 6243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:47.946895 6243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:47.946918 6243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:47.948112 6243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:47.948156 6243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:47.948170 6243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:47.948218 6243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:47.948237 6243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:09:47.948262 6243 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:09:47.948289 6243 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 12:09:47.949743 6243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:41003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:41003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:47.949985 6243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:47.950196 6243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:47.954928 6248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:47.954978 6249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:47.954924 6251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:47.955179 6243 server_base.cc:1034] running on GCE node
I20260619 12:09:47.955469 6243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:47.955744 6243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:47.956913 6243 hybrid_clock.cc:648] HybridClock initialized: now 1781870987956898 us; error 27 us; skew 500 ppm
I20260619 12:09:47.958069 6243 webserver.cc:460] Webserver started at http://127.0.201.62:45803/ using document root <none> and password file <none>
I20260619 12:09:47.958297 6243 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:47.958364 6243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:47.958518 6243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:47.959614 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "a4dfa768d8da46579974a6713821261a"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "7db33472612ee43af570f35b4d039253"
server_key_iv: "f7e64cfca902368ad53007a2b159db9d"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.959964 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "a4dfa768d8da46579974a6713821261a"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "7db33472612ee43af570f35b4d039253"
server_key_iv: "f7e64cfca902368ad53007a2b159db9d"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.961256 6243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:47.962014 6257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:47.962177 6243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:47.962277 6243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "a4dfa768d8da46579974a6713821261a"
format_stamp: "Formatted at 2026-06-19 12:09:47 on dist-test-slave-c910"
server_key: "7db33472612ee43af570f35b4d039253"
server_key_iv: "f7e64cfca902368ad53007a2b159db9d"
server_key_version: "encryptionkey@0"
I20260619 12:09:47.962348 6243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:47.976419 6243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:47.976704 6243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:47.976851 6243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:47.980548 6243 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:41003
I20260619 12:09:47.980583 6309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:41003 every 8 connection(s)
I20260619 12:09:47.980909 6243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:47.981750 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:47.982015 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6243
I20260619 12:09:47.982123 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:47.982295 804 external_mini_cluster.cc:1428] Setting key 57991e584b04ce10df5ad9716729b879
I20260619 12:09:47.984644 6310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a: Bootstrap starting.
I20260619 12:09:47.985289 6310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:47.985567 6310 log.cc:824] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:47.986239 6310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a: No bootstrap required, opened a new log
I20260619 12:09:47.988106 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:47.982769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40054 (local address 127.0.201.62:41003)
0619 12:09:47.982960 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:47.982969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:47.983002 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:09:47.983554 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:47.983562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:47.983634 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:47.983779 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:47.983929 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.985275 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.986070 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:47.986656 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:47.986791 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:47.987431 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:47.987435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:47.987436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:09:47.987643 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:47.987682 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:09:47.987695 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:47.987763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:47.987817 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:09:47.987889 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260619 12:09:47.988709 6310 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4dfa768d8da46579974a6713821261a" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41003 } }
I20260619 12:09:47.988868 6310 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:47.988902 6310 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4dfa768d8da46579974a6713821261a, State: Initialized, Role: FOLLOWER
I20260619 12:09:47.989022 6310 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4dfa768d8da46579974a6713821261a" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41003 } }
I20260619 12:09:47.989091 6310 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:47.989112 6310 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:47.989176 6310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:47.989233 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:09:47.989670 6310 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4dfa768d8da46579974a6713821261a" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41003 } }
I20260619 12:09:47.989749 6310 leader_election.cc:302] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4dfa768d8da46579974a6713821261a; no voters:
I20260619 12:09:47.989921 6310 leader_election.cc:288] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:47.990115 6316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:47.990186 6310 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:47.990303 6316 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [term 1 LEADER]: Becoming Leader. State: Replica: a4dfa768d8da46579974a6713821261a, State: Running, Role: LEADER
I20260619 12:09:47.990396 6316 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4dfa768d8da46579974a6713821261a" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41003 } }
I20260619 12:09:47.990715 6315 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4dfa768d8da46579974a6713821261a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4dfa768d8da46579974a6713821261a" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41003 } } }
I20260619 12:09:47.990729 6317 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4dfa768d8da46579974a6713821261a. Latest consensus state: current_term: 1 leader_uuid: "a4dfa768d8da46579974a6713821261a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4dfa768d8da46579974a6713821261a" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41003 } } }
I20260619 12:09:47.990803 6315 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.990849 6317 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:47.991037 6324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:47.991529 6324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:47.992975 6324 catalog_manager.cc:1347] Generated new cluster ID: 01b98972d9c84ebc82d101ad306287f8
I20260619 12:09:47.993026 6324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:48.028477 6324 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:48.029003 6324 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:48.034508 6324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4dfa768d8da46579974a6713821261a: Generated new TSK 0
I20260619 12:09:48.034705 6324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:48.075395 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:48.040522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.0.201.62:41003)
0619 12:09:48.040587 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:48.040589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:48.040670 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:09:48.040728 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:48.040729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:48.040741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:48.040806 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:09:48.040914 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:48.041632 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:48.042365 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:48.042667 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:48.042738 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:48.042883 (+ 145us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:09:48.075109 (+ 32226us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 12:09:48.075315 (+ 206us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40062: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 12:09:48.075472 6313 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40062: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 12:09:48.075497 6339 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:48.040448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41003 (local address 127.0.0.1:40062)
0619 12:09:48.040610 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:48.040619 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:48.040666 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:48.040811 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:48.040812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:48.040814 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:09:48.040861 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:48.040862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:48.041709 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:48.041710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:48.042285 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:48.042286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:48.042749 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:48.042750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:48.042787 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:48.042791 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:09:48.075286 (+ 32495us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 12:09:48.075394 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.201.62:41003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260619 12:09:48.075865 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6243
2026-06-19T12:09:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (232 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:48 dist-test-slave-c910 krb5kdc[6351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:48 dist-test-slave-c910 krb5kdc[6351](info): set up 2 sockets
Jun 19 12:09:48 dist-test-slave-c910 krb5kdc[6351](info): commencing operation
krb5kdc: starting...
W20260619 12:09:49.271363 804 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:49.383561 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:35919
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:35919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:49.463285 6364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:49.463516 6364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:49.463539 6364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:49.464605 6364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:49.464638 6364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:49.464653 6364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:49.464664 6364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:49.464676 6364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:49.466084 6364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:35919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:35919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:49.466259 6364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:49.466413 6364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:49.471067 6372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.471071 6369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.471138 6370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:49.471446 6364 server_base.cc:1034] running on GCE node
I20260619 12:09:49.471676 6364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:49.471966 6364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:49.473126 6364 hybrid_clock.cc:648] HybridClock initialized: now 1781870989473120 us; error 37 us; skew 500 ppm
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:49.478541 6364 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:49.478962 6364 webserver.cc:460] Webserver started at http://127.0.201.62:44211/ using document root <none> and password file <none>
I20260619 12:09:49.479233 6364 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:49.479318 6364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:49.479467 6364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:49.480497 6364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "05e9dbf98ae54db68643e91abd25200e"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "9c09a785e8792c761c2b086e3b3b446a"
server_key_iv: "86e7f72a98fda49816e8c91868063530"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.480850 6364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "05e9dbf98ae54db68643e91abd25200e"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "9c09a785e8792c761c2b086e3b3b446a"
server_key_iv: "86e7f72a98fda49816e8c91868063530"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.482143 6364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:49.482823 6379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.482980 6364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:49.483076 6364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "05e9dbf98ae54db68643e91abd25200e"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "9c09a785e8792c761c2b086e3b3b446a"
server_key_iv: "86e7f72a98fda49816e8c91868063530"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.483160 6364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:49.492339 6364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:49.494004 6364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:49.494160 6364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:49.498059 6364 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:35919
I20260619 12:09:49.498086 6431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:35919 every 8 connection(s)
I20260619 12:09:49.498566 6364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:49.499186 6432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:49.501371 6432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e: Bootstrap starting.
I20260619 12:09:49.501919 6432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:49.502197 6432 log.cc:824] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:49.502830 6432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e: No bootstrap required, opened a new log
I20260619 12:09:49.504335 6432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e9dbf98ae54db68643e91abd25200e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35919 } }
I20260619 12:09:49.504468 6432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:49.504490 6432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e9dbf98ae54db68643e91abd25200e, State: Initialized, Role: FOLLOWER
I20260619 12:09:49.504637 6432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e9dbf98ae54db68643e91abd25200e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35919 } }
I20260619 12:09:49.504707 6432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:49.504729 6432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:49.504776 6432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:49.505625 6432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e9dbf98ae54db68643e91abd25200e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35919 } }
I20260619 12:09:49.505708 6432 leader_election.cc:302] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05e9dbf98ae54db68643e91abd25200e; no voters:
I20260619 12:09:49.505880 6432 leader_election.cc:288] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:49.505956 6435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:49.506112 6435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [term 1 LEADER]: Becoming Leader. State: Replica: 05e9dbf98ae54db68643e91abd25200e, State: Running, Role: LEADER
I20260619 12:09:49.506147 6432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:49.506206 6435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e9dbf98ae54db68643e91abd25200e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35919 } }
I20260619 12:09:49.506587 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e9dbf98ae54db68643e91abd25200e. Latest consensus state: current_term: 1 leader_uuid: "05e9dbf98ae54db68643e91abd25200e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e9dbf98ae54db68643e91abd25200e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35919 } } }
I20260619 12:09:49.506584 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05e9dbf98ae54db68643e91abd25200e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e9dbf98ae54db68643e91abd25200e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35919 } } }
I20260619 12:09:49.506713 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:49.506726 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:49.506953 6444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:49.507490 6444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:49.507922 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6364
I20260619 12:09:49.508026 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:49.508198 804 external_mini_cluster.cc:1428] Setting key b6238dafc253065c3601224411116e40
I20260619 12:09:49.509827 6444 catalog_manager.cc:1347] Generated new cluster ID: bba5012895c440058ab212a40787f104
I20260619 12:09:49.509881 6444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:49.525039 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:49.508646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.0.201.62:35919)
0619 12:09:49.508785 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:49.508794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:49.508828 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:09:49.509000 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:49.509008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:49.509064 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:49.509223 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:49.515451 (+ 6228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.516901 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.517906 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.518565 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.518676 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:49.521430 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:49.521434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:49.521434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:49.521447 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:49.523329 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:49.523726 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:49.523729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:49.523729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:49.523849 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:49.524048 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:49.524049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:49.524049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:49.524332 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:49.524470 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:49.524597 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 12:09:49.524818 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260619 12:09:49.538187 6444 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:49.539078 6444 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:49.551862 6444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05e9dbf98ae54db68643e91abd25200e: Generated new TSK 0
I20260619 12:09:49.552094 6444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:09:49.587631 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35919
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:49.667343 6458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:49.667547 6458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:49.667569 6458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:49.668704 6458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:49.668744 6458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:49.668769 6458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:49.670189 6458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:49.670411 6458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:49.670644 6458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:49.675477 6466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.675480 6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:49.675655 6458 server_base.cc:1034] running on GCE node
W20260619 12:09:49.675480 6464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:49.675963 6458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:49.676224 6458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:49.677407 6458 hybrid_clock.cc:648] HybridClock initialized: now 1781870989677372 us; error 47 us; skew 500 ppm
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:49.679904 6458 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:09:49.680315 6458 webserver.cc:460] Webserver started at http://127.0.201.1:37657/ using document root <none> and password file <none>
I20260619 12:09:49.680544 6458 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:49.680612 6458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:49.680758 6458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:49.681831 6458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "97e338b84e0249029f8920945a7eaec1"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "d67a99d4d462efbbbe7c75f474a842b7"
server_key_iv: "09e2e09b6cc61af23aa3ed38e0ad1017"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.682183 6458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "97e338b84e0249029f8920945a7eaec1"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "d67a99d4d462efbbbe7c75f474a842b7"
server_key_iv: "09e2e09b6cc61af23aa3ed38e0ad1017"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.683450 6458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:09:49.684158 6473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.684412 6458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:49.684505 6458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "97e338b84e0249029f8920945a7eaec1"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "d67a99d4d462efbbbe7c75f474a842b7"
server_key_iv: "09e2e09b6cc61af23aa3ed38e0ad1017"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.684583 6458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:49.703426 6458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:49.706490 6458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:49.706723 6458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:49.706974 6458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:49.707415 6458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:49.707455 6458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.707499 6458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:49.707532 6458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.713388 6458 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:44861
I20260619 12:09:49.713450 6586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:44861 every 8 connection(s)
I20260619 12:09:49.713770 6458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:49.722275 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6458
I20260619 12:09:49.722398 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:49.722589 804 external_mini_cluster.cc:1428] Setting key fc50b3fefe48c59194565fde5e82689d
I20260619 12:09:49.722767 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:49.714506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:59843 (local address 127.0.201.62:35919)
0619 12:09:49.714559 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:49.714561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:49.714759 (+ 198us) server_negotiation.cc:407] Connection header received
0619 12:09:49.715753 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:49.715754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:49.715766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:49.715830 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:49.716405 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.716889 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.717869 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.718055 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.718104 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:49.720083 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:49.720084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:49.720084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:49.720090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:49.721503 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:49.721835 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:49.721835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:49.721836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:49.721883 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:49.722080 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:49.722081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:49.722081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:49.722174 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:49.722252 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:49.722595 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 12:09:49.722678 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:09:49.722954 6589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:49.714470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35919 (local address 127.0.201.1:59843)
0619 12:09:49.714643 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:49.714684 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:49.715662 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:49.715894 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:49.715898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:49.716149 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:49.716332 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:49.716335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.716968 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.716968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:49.717776 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:49.717778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.718123 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.718124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:49.718220 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:49.718673 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:49.718681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:49.719996 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:49.721571 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:49.721575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:49.721577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:49.721772 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:49.721939 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:49.721939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:49.721940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:49.722028 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:49.722258 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:49.722264 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:49.722450 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 12:09:49.722559 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:09:49.722709 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260619 12:09:49.723498 6587 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35919
I20260619 12:09:49.723599 6587 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:49.723764 6587 heartbeater.cc:511] Master 127.0.201.62:35919 requested a full tablet report, sending...
I20260619 12:09:49.724291 6396 ts_manager.cc:194] Registered new tserver with Master: 97e338b84e0249029f8920945a7eaec1 (127.0.201.1:44861)
I20260619 12:09:49.725262 6396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:59843
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:09:49.747736 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35919
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:49.827271 6592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:49.827490 6592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:49.827513 6592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:49.828604 6592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:49.828635 6592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:49.828661 6592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:49.830000 6592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:49.830164 6592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:49.830324 6592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:49.835105 6597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.835111 6600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.835119 6598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:49.835386 6592 server_base.cc:1034] running on GCE node
I20260619 12:09:49.835678 6592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:49.835961 6592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:49.837134 6592 hybrid_clock.cc:648] HybridClock initialized: now 1781870989837120 us; error 48 us; skew 500 ppm
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:49.839788 6592 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:09:49.840245 6592 webserver.cc:460] Webserver started at http://127.0.201.2:34605/ using document root <none> and password file <none>
I20260619 12:09:49.840517 6592 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:49.840591 6592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:49.840736 6592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:49.841799 6592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "1ff81b036460443a9d389e1712fff4da"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "707f50a537bd8b44f1f9818085c598f5"
server_key_iv: "1c83154bfa4b9bece5a808f231f46399"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.842146 6592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "1ff81b036460443a9d389e1712fff4da"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "707f50a537bd8b44f1f9818085c598f5"
server_key_iv: "1c83154bfa4b9bece5a808f231f46399"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.843426 6592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:49.844071 6607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.844225 6592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:49.844326 6592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "1ff81b036460443a9d389e1712fff4da"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "707f50a537bd8b44f1f9818085c598f5"
server_key_iv: "1c83154bfa4b9bece5a808f231f46399"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.844406 6592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:49.861300 6592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:49.863667 6592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:49.863775 6592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:49.863936 6592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:49.864241 6592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:49.864307 6592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.864347 6592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:49.864377 6592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.870100 6592 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:33923
I20260619 12:09:49.870172 6720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:33923 every 8 connection(s)
I20260619 12:09:49.870482 6592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:49.872258 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6592
I20260619 12:09:49.872370 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:49.872555 804 external_mini_cluster.cc:1428] Setting key 5a557a8f1d97a16edbd3abaaafefb2df
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:49.880817 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:49.871225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:47805 (local address 127.0.201.62:35919)
0619 12:09:49.871318 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:49.871320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:49.871485 (+ 165us) server_negotiation.cc:407] Connection header received
0619 12:09:49.872664 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:49.872666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:49.872677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:49.872747 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:49.873546 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.874031 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.875234 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.875540 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.875607 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:49.877888 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:49.877889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:49.877890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:49.877898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:49.879469 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:49.879846 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:49.879848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:49.879848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:49.879908 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:49.880135 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:49.880136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:49.880137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:49.880282 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:49.880357 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:49.880642 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 12:09:49.880728 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:09:49.880898 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:49.871170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35919 (local address 127.0.201.2:47805)
0619 12:09:49.871367 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:49.871401 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:49.872544 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:49.872833 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:49.872837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:49.873158 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:49.873430 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:49.873437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.874121 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.874122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:49.875094 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:49.875096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:49.875633 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:49.875634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:49.875738 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:49.876241 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:49.876249 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:49.877788 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:49.879543 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:49.879549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:49.879550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:49.879767 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:49.879975 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:49.879975 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:49.879976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:49.880067 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:49.880372 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:49.880378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:49.880490 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 12:09:49.880584 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 12:09:49.880697 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
I20260619 12:09:49.881348 6721 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35919
I20260619 12:09:49.881457 6721 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:49.881634 6721 heartbeater.cc:511] Master 127.0.201.62:35919 requested a full tablet report, sending...
I20260619 12:09:49.882035 6396 ts_manager.cc:194] Registered new tserver with Master: 1ff81b036460443a9d389e1712fff4da (127.0.201.2:33923)
I20260619 12:09:49.882378 6396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:47805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:09:49.897626 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35919
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:49.977245 6726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:49.977487 6726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:49.977527 6726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:49.978688 6726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:49.978754 6726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:49.978799 6726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:49.980393 6726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:49.980654 6726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:49.980893 6726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:49.985828 6734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.985919 6732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:49.985839 6731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:49.986132 6726 server_base.cc:1034] running on GCE node
I20260619 12:09:49.986340 6726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:49.986527 6726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:49.987710 6726 hybrid_clock.cc:648] HybridClock initialized: now 1781870989987696 us; error 28 us; skew 500 ppm
Jun 19 12:09:49 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:49.990381 6726 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:09:49.990856 6726 webserver.cc:460] Webserver started at http://127.0.201.3:41231/ using document root <none> and password file <none>
I20260619 12:09:49.991096 6726 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:49.991164 6726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:49.991324 6726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:49.992329 6726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "a703e94a30004c3bb08cc9075ac484b5"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "bbdc520e4637b4f0b8b170ad8d665a95"
server_key_iv: "d15f7594906a97e7dc7023759e81e51e"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.992686 6726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "a703e94a30004c3bb08cc9075ac484b5"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "bbdc520e4637b4f0b8b170ad8d665a95"
server_key_iv: "d15f7594906a97e7dc7023759e81e51e"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.993939 6726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:09:49.994604 6741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:49.994772 6726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:49.994868 6726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "a703e94a30004c3bb08cc9075ac484b5"
format_stamp: "Formatted at 2026-06-19 12:09:49 on dist-test-slave-c910"
server_key: "bbdc520e4637b4f0b8b170ad8d665a95"
server_key_iv: "d15f7594906a97e7dc7023759e81e51e"
server_key_version: "encryptionkey@0"
I20260619 12:09:49.994951 6726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:50.006620 6726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:50.009032 6726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:50.009140 6726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:50.009302 6726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:50.009536 6726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:50.009567 6726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:50.009586 6726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:50.009608 6726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:50.015530 6726 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:35235
I20260619 12:09:50.015599 6854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:35235 every 8 connection(s)
I20260619 12:09:50.015914 6726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:50.022220 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6726
I20260619 12:09:50.022305 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:09:50.022535 804 external_mini_cluster.cc:1428] Setting key 91f678246c1d9eda929b5a87a74c70bf
I20260619 12:09:50.025923 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:50.016674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:37741 (local address 127.0.201.62:35919)
0619 12:09:50.016741 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.016743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.016961 (+ 218us) server_negotiation.cc:407] Connection header received
0619 12:09:50.018112 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.018114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.018126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.018185 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.018825 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.019423 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.020431 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.020636 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.020722 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:50.023064 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:50.023066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:50.023066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:50.023074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:50.024630 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.025051 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.025052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.025053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.025133 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.025350 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.025351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.025351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.025450 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:50.025512 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:50.025787 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 12:09:50.025852 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:09:50.026156 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:50.016651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35919 (local address 127.0.201.3:37741)
0619 12:09:50.016824 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:50.016877 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:50.017823 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:50.018257 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:50.018261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:50.018529 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:50.018718 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:50.018722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.019515 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.019516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:50.020311 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:50.020313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.020730 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.020731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:50.020959 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:50.021402 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:50.021419 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:50.022905 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:50.024710 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:50.024716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:50.024718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:50.024976 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:50.025197 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:50.025198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:50.025198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:50.025294 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:50.025541 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:50.025547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:50.025667 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 12:09:50.025766 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:09:50.025922 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260619 12:09:50.026559 6855 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35919
I20260619 12:09:50.026638 6855 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:50.026826 6855 heartbeater.cc:511] Master 127.0.201.62:35919 requested a full tablet report, sending...
I20260619 12:09:50.027174 6396 ts_manager.cc:194] Registered new tserver with Master: a703e94a30004c3bb08cc9075ac484b5 (127.0.201.3:35235)
I20260619 12:09:50.027617 6396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:37741
I20260619 12:09:50.034614 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/test-admin.keytab.
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.048923 804 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 12:09:50.050431 6862 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:50.049698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.0.201.1:44861)
0619 12:09:50.049861 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.049866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.049884 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:09:50.049907 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.049909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.049922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.050001 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.050370 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42522: BlockingRecv error: recv got EOF from 127.0.0.1:42522 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
W20260619 12:09:50.050624 804 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.051563 804 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.052687 804 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:50.060451 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:50.053151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.0.201.1:44861)
0619 12:09:50.053191 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.053197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.053206 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:50.053247 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.053247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.053256 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.053300 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.053637 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.054384 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.055077 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.055549 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.055607 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:50.057914 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:50.057921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:50.057921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:50.057927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:50.059419 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.059719 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.059722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.059722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.059773 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.059934 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.059935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.059935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.060059 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:50.060159 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:50.060222 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 12:09:50.060281 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 12:09:50.062278 6862 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:50.061677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.0.201.1:44861)
0619 12:09:50.061736 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.061738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.061775 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:09:50.061868 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.061869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.061880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.061933 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.062226 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42538: BlockingRecv error: recv got EOF from 127.0.0.1:42538 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 12:09:50.062333 804 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.063301 804 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.064170 804 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:50.071035 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:50.064627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.0.201.1:44861)
0619 12:09:50.064673 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.064675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.064687 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:09:50.064758 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.064759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.064765 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.064811 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.065157 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.065617 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.066278 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.066456 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.066512 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:50.068590 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:50.068591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:50.068592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:50.068598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:50.069963 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.070333 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.070334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.070335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.070395 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.070589 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.070590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.070590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.070682 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:50.070780 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:50.070873 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 12:09:50.070943 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 12:09:50.072377 6862 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:50.071939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.0.201.1:44861)
0619 12:09:50.071969 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.071971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.072000 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:09:50.072053 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.072053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.072060 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.072102 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.072337 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42560: BlockingRecv error: recv got EOF from 127.0.0.1:42560 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260619 12:09:50.072475 804 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781870963166527-804-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.073387 804 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:50.074187 804 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:09:50.080964 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:50.074631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.0.201.1:44861)
0619 12:09:50.074687 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:50.074689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:50.074719 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:09:50.074869 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:50.074870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:50.074880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:50.074936 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:50.075337 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.075772 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.076462 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:50.076648 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:50.076728 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:50.078632 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:50.078633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:50.078634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:50.078641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:50.079872 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.080207 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.080208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.080208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.080271 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:50.080492 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:50.080492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:50.080493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:50.080586 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:50.080684 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:50.080789 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:09:50.080858 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:09:50.081633 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6458
I20260619 12:09:50.086201 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6592
I20260619 12:09:50.090647 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6726
I20260619 12:09:50.096246 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6364
2026-06-19T12:09:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1850 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6875](info): set up 2 sockets
Jun 19 12:09:50 dist-test-slave-c910 krb5kdc[6875](info): commencing operation
krb5kdc: starting...
W20260619 12:09:51.131453 804 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab.
Entry for principal oryx/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab.
I20260619 12:09:51.240012 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:35305
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab
--principal=oryx/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:35305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:51.318105 6888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:51.318342 6888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:51.318388 6888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:51.319891 6888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:51.319965 6888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:51.319991 6888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:51.320020 6888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:51.320050 6888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:51.321650 6888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:35305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab
--principal=oryx/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:35305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:51.321944 6888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:51.322168 6888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:51.326790 6896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.326869 6894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.326936 6893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:51.327095 6888 server_base.cc:1034] running on GCE node
I20260619 12:09:51.327353 6888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:51.327603 6888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:51.328765 6888 hybrid_clock.cc:648] HybridClock initialized: now 1781870991328751 us; error 25 us; skew 500 ppm
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:51.331099 6888 init.cc:377] Logged in from keytab as oryx/127.0.201.62@KRBTEST.COM (short username oryx)
I20260619 12:09:51.331568 6888 webserver.cc:460] Webserver started at http://127.0.201.62:40227/ using document root <none> and password file <none>
I20260619 12:09:51.331749 6888 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:51.331789 6888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:51.331884 6888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:51.332887 6888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "1a4785b9c63e4fdba05b1257ea44c8af"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4b62a096e211bb0bb211fce4f88403bf"
server_key_iv: "223b1f167eb6082f0e26612caa1a624f"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.333180 6888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "1a4785b9c63e4fdba05b1257ea44c8af"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4b62a096e211bb0bb211fce4f88403bf"
server_key_iv: "223b1f167eb6082f0e26612caa1a624f"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.334378 6888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:51.335021 6903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.335167 6888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:51.335269 6888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "1a4785b9c63e4fdba05b1257ea44c8af"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4b62a096e211bb0bb211fce4f88403bf"
server_key_iv: "223b1f167eb6082f0e26612caa1a624f"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.335321 6888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:51.387169 6888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:51.389541 6888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:51.389655 6888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:51.393484 6888 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:35305
I20260619 12:09:51.393544 6955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:35305 every 8 connection(s)
I20260619 12:09:51.393899 6888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:51.394763 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6888
I20260619 12:09:51.394884 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:51.395013 6956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:51.395108 804 external_mini_cluster.cc:1428] Setting key 61488abcc83b9121983bd6ced2ae2995
I20260619 12:09:51.398475 6956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Bootstrap starting.
I20260619 12:09:51.399080 6956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:51.399386 6956 log.cc:824] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:51.400053 6956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: No bootstrap required, opened a new log
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:51.401832 6956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } }
I20260619 12:09:51.401993 6956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:51.402056 6956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a4785b9c63e4fdba05b1257ea44c8af, State: Initialized, Role: FOLLOWER
I20260619 12:09:51.402180 6956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } }
I20260619 12:09:51.402283 6956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:51.402326 6956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:51.402372 6956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:51.402864 6956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } }
I20260619 12:09:51.402976 6956 leader_election.cc:302] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a4785b9c63e4fdba05b1257ea44c8af; no voters:
I20260619 12:09:51.403141 6956 leader_election.cc:288] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:51.403172 6961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:51.403316 6961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 LEADER]: Becoming Leader. State: Replica: 1a4785b9c63e4fdba05b1257ea44c8af, State: Running, Role: LEADER
I20260619 12:09:51.403400 6961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } }
I20260619 12:09:51.403424 6956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:51.404937 6963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } }
I20260619 12:09:51.405210 6963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
W20260619 12:09:51.405278 6975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:51.405323 6975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:51.405053 6962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a4785b9c63e4fdba05b1257ea44c8af. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } }
I20260619 12:09:51.405459 6962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:51.405896 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.395660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.0.201.62:35305)
0619 12:09:51.395855 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:51.395865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:51.395901 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:09:51.396334 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:51.396343 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:51.396404 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:51.396555 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:51.397116 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.398457 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.399302 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.399772 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.399901 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.402151 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:51.402155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:51.402156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:51.402168 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:51.404042 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.404389 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.404392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.404393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.404482 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.404714 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.404715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.404715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.405128 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:51.405274 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:51.405670 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 12:09:51.405764 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260619 12:09:51.406064 6978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:51.406220 6978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:51.408632 6978 catalog_manager.cc:1347] Generated new cluster ID: ff452963a7924aa88c4c69f334abe857
I20260619 12:09:51.408710 6978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:51.425203 6978 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:51.425766 6978 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:51.433561 6978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Generated new TSK 0
I20260619 12:09:51.433755 6978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab.
Entry for principal oryx/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab.
I20260619 12:09:51.459164 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35305
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:51.537353 6982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:51.537554 6982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:51.537575 6982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:51.538832 6982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:51.538874 6982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:51.538900 6982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:51.540334 6982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:51.540563 6982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:51.540776 6982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:51.545542 6987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.545537 6990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:51.545856 6982 server_base.cc:1034] running on GCE node
W20260619 12:09:51.545571 6988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:51.546154 6982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:51.546386 6982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:51.547544 6982 hybrid_clock.cc:648] HybridClock initialized: now 1781870991547540 us; error 35 us; skew 500 ppm
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:51.550114 6982 init.cc:377] Logged in from keytab as oryx/127.0.201.1@KRBTEST.COM (short username oryx)
I20260619 12:09:51.550539 6982 webserver.cc:460] Webserver started at http://127.0.201.1:43101/ using document root <none> and password file <none>
I20260619 12:09:51.550712 6982 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:51.550750 6982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:51.550832 6982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:51.551815 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "5b3ed0457e54445c8ddf7e804eee797c"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "24c7043db5fc309be298192e37a9b088"
server_key_iv: "f281cd82145bd3d3d1cdf7de3e453668"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.552110 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "5b3ed0457e54445c8ddf7e804eee797c"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "24c7043db5fc309be298192e37a9b088"
server_key_iv: "f281cd82145bd3d3d1cdf7de3e453668"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.553316 6982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:51.554008 6997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.554212 6982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:51.554278 6982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "5b3ed0457e54445c8ddf7e804eee797c"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "24c7043db5fc309be298192e37a9b088"
server_key_iv: "f281cd82145bd3d3d1cdf7de3e453668"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.554320 6982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:51.567889 6982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:51.570067 6982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:51.570173 6982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:51.570329 6982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:51.570643 6982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:51.570683 6982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.570725 6982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:51.570750 6982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.576421 6982 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:33203
I20260619 12:09:51.576478 7110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:33203 every 8 connection(s)
I20260619 12:09:51.576762 6982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:51.583530 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 6982
I20260619 12:09:51.583626 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:51.583863 804 external_mini_cluster.cc:1428] Setting key 0eed2e179fd61ab1c8b233041d839aa2
I20260619 12:09:51.585601 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.577363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:59233 (local address 127.0.201.62:35305)
0619 12:09:51.577401 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:51.577403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:51.577607 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:09:51.578043 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:51.578044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:51.578054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:51.578105 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:51.578950 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.579529 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.580446 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.580641 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.580695 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.582652 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:51.582654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:51.582654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:51.582660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:51.584247 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.584682 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.584683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.584683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.584745 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.584987 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.584988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.584988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.585101 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:51.585175 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:51.585436 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 12:09:51.585514 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 12:09:51.585951 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.577339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:59233)
0619 12:09:51.577480 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:51.577512 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:51.577957 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:51.578154 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:51.578157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:51.578685 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:51.578862 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:51.578866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.579600 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.579600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:51.580365 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:51.580367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.580712 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.580713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:51.580802 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.581285 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:51.581293 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:51.582536 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:51.584319 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:51.584324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:51.584326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:51.584599 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:51.584810 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:51.584810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:51.584811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:51.584915 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:51.585165 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:51.585171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:51.585304 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 12:09:51.585409 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:09:51.585551 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260619 12:09:51.586364 7111 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35305
I20260619 12:09:51.586473 7111 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:51.586673 7111 heartbeater.cc:511] Master 127.0.201.62:35305 requested a full tablet report, sending...
I20260619 12:09:51.587157 6920 ts_manager.cc:194] Registered new tserver with Master: 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203)
I20260619 12:09:51.587790 6920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:59233
WARNING: no policy specified for oryx/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab.
Entry for principal oryx/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab.
I20260619 12:09:51.606729 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35305
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:51.684651 7116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:51.684835 7116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:51.684859 7116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:51.685935 7116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:51.685969 7116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:51.685994 7116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:51.687362 7116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:51.687523 7116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:51.687677 7116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:51.692304 7122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.692288 7121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.692279 7124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:51.692509 7116 server_base.cc:1034] running on GCE node
I20260619 12:09:51.692672 7116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:51.692845 7116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:51.693974 7116 hybrid_clock.cc:648] HybridClock initialized: now 1781870991693963 us; error 27 us; skew 500 ppm
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:51.696350 7116 init.cc:377] Logged in from keytab as oryx/127.0.201.2@KRBTEST.COM (short username oryx)
I20260619 12:09:51.696727 7116 webserver.cc:460] Webserver started at http://127.0.201.2:44963/ using document root <none> and password file <none>
I20260619 12:09:51.696954 7116 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:51.697018 7116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:51.697161 7116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:51.698110 7116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "2c149b0fb6b94ede882ed006d1442da6"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "e5ef9a4824797abba64b6b01294c50e5"
server_key_iv: "8c59507ab0f95259aff6405dbfc4ed95"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.698501 7116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "2c149b0fb6b94ede882ed006d1442da6"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "e5ef9a4824797abba64b6b01294c50e5"
server_key_iv: "8c59507ab0f95259aff6405dbfc4ed95"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.700192 7116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:51.701162 7131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.701366 7116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:51.701467 7116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "2c149b0fb6b94ede882ed006d1442da6"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "e5ef9a4824797abba64b6b01294c50e5"
server_key_iv: "8c59507ab0f95259aff6405dbfc4ed95"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.701545 7116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:51.712774 7116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:51.714385 7116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:51.714511 7116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:51.714674 7116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:51.714965 7116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:51.715021 7116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.715061 7116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:51.715087 7116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.720645 7116 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44847
I20260619 12:09:51.720708 7244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44847 every 8 connection(s)
I20260619 12:09:51.721016 7116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:51.729249 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.721669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:57103 (local address 127.0.201.62:35305)
0619 12:09:51.721732 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:51.721734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:51.721895 (+ 161us) server_negotiation.cc:407] Connection header received
0619 12:09:51.722325 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:51.722326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:51.722335 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:51.722386 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:51.723278 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.723757 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.724714 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.724887 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.724935 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.726901 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:51.726902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:51.726903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:51.726909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:51.728172 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.728529 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.728529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.728530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.728574 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.728780 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.728781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.728781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.728872 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:51.728919 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:51.729134 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 12:09:51.729190 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:09:51.729406 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.721634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:57103)
0619 12:09:51.721781 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:51.721819 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:51.722248 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:51.722434 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:51.722437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:51.722958 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:51.723127 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:51.723131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.723839 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.723839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:51.724642 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:51.724644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.724955 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.724956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:51.725054 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.725461 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:51.725468 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:51.726823 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:51.728248 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:51.728254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:51.728255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:51.728464 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:51.728630 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:51.728630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:51.728630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:51.728728 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:51.728924 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:51.728928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:51.729028 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 12:09:51.729111 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:09:51.729219 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260619 12:09:51.729918 7245 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35305
I20260619 12:09:51.730002 7245 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:51.730123 7245 heartbeater.cc:511] Master 127.0.201.62:35305 requested a full tablet report, sending...
I20260619 12:09:51.730472 6920 ts_manager.cc:194] Registered new tserver with Master: 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847)
I20260619 12:09:51.730732 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 7116
I20260619 12:09:51.730834 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:51.730857 6920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:57103
I20260619 12:09:51.731027 804 external_mini_cluster.cc:1428] Setting key cfc5b0620e5350918c61412b03667acf
WARNING: no policy specified for oryx/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab.
Entry for principal oryx/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab.
I20260619 12:09:51.753639 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35305
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:51.831769 7250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:51.831981 7250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:51.832016 7250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:51.833098 7250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:51.833164 7250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:51.833207 7250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:51.834652 7250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:35305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:51.834909 7250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:51.835121 7250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:51.839919 7258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.839993 7255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:51.839907 7256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:51.840138 7250 server_base.cc:1034] running on GCE node
I20260619 12:09:51.840359 7250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:51.840596 7250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:51.841753 7250 hybrid_clock.cc:648] HybridClock initialized: now 1781870991841751 us; error 36 us; skew 500 ppm
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:51.844028 7250 init.cc:377] Logged in from keytab as oryx/127.0.201.3@KRBTEST.COM (short username oryx)
I20260619 12:09:51.844434 7250 webserver.cc:460] Webserver started at http://127.0.201.3:41877/ using document root <none> and password file <none>
I20260619 12:09:51.844659 7250 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:51.844725 7250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:51.844871 7250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:51.845830 7250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "953edcf192924724ab15ed83aa6156bf"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4d8bdc0081e3ed71aedfb6cf9b1ef648"
server_key_iv: "4cad231b02c26df0eab8b65aed382c34"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.846179 7250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "953edcf192924724ab15ed83aa6156bf"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4d8bdc0081e3ed71aedfb6cf9b1ef648"
server_key_iv: "4cad231b02c26df0eab8b65aed382c34"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.847491 7250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:51.848220 7265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.848387 7250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:51.848480 7250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "953edcf192924724ab15ed83aa6156bf"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4d8bdc0081e3ed71aedfb6cf9b1ef648"
server_key_iv: "4cad231b02c26df0eab8b65aed382c34"
server_key_version: "encryptionkey@0"
I20260619 12:09:51.848557 7250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:51.871184 7250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:51.873057 7250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:51.873214 7250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:51.873430 7250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:51.873716 7250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:51.873766 7250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.873798 7250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:51.873823 7250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:51.879525 7250 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:42477
I20260619 12:09:51.879568 7378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:42477 every 8 connection(s)
I20260619 12:09:51.879856 7250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:09:51 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:51.888234 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 7250
I20260619 12:09:51.888331 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:51.888563 804 external_mini_cluster.cc:1428] Setting key 67a1f62aabc9c75b84f59ce5b134dc62
I20260619 12:09:51.889954 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.880573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:56475 (local address 127.0.201.62:35305)
0619 12:09:51.880652 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:51.880654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:51.880811 (+ 157us) server_negotiation.cc:407] Connection header received
0619 12:09:51.881263 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:51.881264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:51.881276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:51.881336 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:51.882219 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.882720 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.883779 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.884015 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.884076 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.886538 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:51.886540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:51.886540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:51.886548 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:51.888199 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.888743 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.888745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.888746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.888829 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:51.889202 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:51.889203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:51.889204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:51.889360 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:51.889430 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:51.889778 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 12:09:51.889860 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:09:51.890141 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:51.880520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:56475)
0619 12:09:51.880697 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:51.880730 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:51.881175 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:51.881407 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:51.881421 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:51.881952 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:51.882129 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:51.882133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.882799 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.882799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:51.883662 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:51.883664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:51.884101 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:51.884102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:51.884218 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:51.884705 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:51.884715 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:51.886422 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:51.888343 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:51.888349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:51.888351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:51.888620 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:51.888932 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:51.888933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:51.888934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:51.889077 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:51.889436 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:51.889442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:51.889609 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 12:09:51.889754 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 12:09:51.889911 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260619 12:09:51.890604 7379 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35305
I20260619 12:09:51.890726 7379 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:51.890900 7379 heartbeater.cc:511] Master 127.0.201.62:35305 requested a full tablet report, sending...
I20260619 12:09:51.891337 6920 ts_manager.cc:194] Registered new tserver with Master: 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477)
I20260619 12:09:51.891674 6920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:56475
I20260619 12:09:51.900318 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.201.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.61.keytab.
Entry for principal oryx/127.0.201.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.61.keytab.
I20260619 12:09:51.924065 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:37357
--webserver_interface=127.0.201.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.61.keytab
--principal=oryx/127.0.201.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:35305,127.0.201.61:37357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:52.004139 7384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:52.004418 7384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:52.004458 7384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:52.005627 7384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:52.005697 7384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:52.005735 7384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:52.005769 7384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:52.005796 7384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:52.007369 7384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:35305,127.0.201.61:37357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.61.keytab
--principal=oryx/127.0.201.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:37357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.201.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
W20260619 12:09:52.007507 7384 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.201.62:35305,127.0.201.61:37357'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 12:09:52.007666 7384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:52.007893 7384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.012652 7389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.012669 7392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.012862 7384 server_base.cc:1034] running on GCE node
W20260619 12:09:52.012691 7390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.013144 7384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.013410 7384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.014571 7384 hybrid_clock.cc:648] HybridClock initialized: now 1781870992014568 us; error 36 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.017011 7384 init.cc:377] Logged in from keytab as oryx/127.0.201.61@KRBTEST.COM (short username oryx)
I20260619 12:09:52.017465 7384 webserver.cc:460] Webserver started at http://127.0.201.61:41063/ using document root <none> and password file <none>
I20260619 12:09:52.017699 7384 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.017766 7384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.017920 7384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:52.018893 7384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/instance:
uuid: "404e21e3716344df971cf67bcbb55655"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "dc92c600339ce329037931d48153cef6"
server_key_iv: "2cbc412d18613e0d0cb4af267a1e4b60"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.019302 7384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal/instance:
uuid: "404e21e3716344df971cf67bcbb55655"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "dc92c600339ce329037931d48153cef6"
server_key_iv: "2cbc412d18613e0d0cb4af267a1e4b60"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.020561 7384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:09:52.021324 7399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.021503 7384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:52.021596 7384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
uuid: "404e21e3716344df971cf67bcbb55655"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "dc92c600339ce329037931d48153cef6"
server_key_iv: "2cbc412d18613e0d0cb4af267a1e4b60"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.021679 7384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.034183 7384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.036017 7384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:52.036165 7384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.040287 7384 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.61:37357
I20260619 12:09:52.040325 7451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.61:37357 every 8 connection(s)
I20260619 12:09:52.040668 7384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
I20260619 12:09:52.041559 7452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:52.042980 7452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:52.048249 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 7384
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.61@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:52.052359 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.043670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.0.201.62:35305)
0619 12:09:52.043744 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.043747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.043909 (+ 162us) server_negotiation.cc:407] Connection header received
0619 12:09:52.044325 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.044326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.044336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.044391 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.044942 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.045417 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.046327 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.046536 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.046589 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.049043 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.049045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.049045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.049053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.050726 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.051258 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.051259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.051260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.051325 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.051621 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.051621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.051622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.051751 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.051801 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.052152 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.052265 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:09:52.052557 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.043630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:58656)
0619 12:09:52.043796 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.043831 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.044244 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.044470 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.044473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.044696 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.044869 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.044873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.045488 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.045489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.046256 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.046258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.046606 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.046606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.046708 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.047150 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.047158 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.048924 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.050814 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.050820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.050822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.051104 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.051401 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.051402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.051402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.051549 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.051816 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.051821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.051971 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.052110 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.052269 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260619 12:09:52.053189 7452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.61@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
I20260619 12:09:52.058089 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.048839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.0.201.61:37357)
0619 12:09:52.048980 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.048986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.049008 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:09:52.049085 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.049088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.049102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.049177 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.049937 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.050972 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.051994 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.052258 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.052358 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.054877 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.054883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.054884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.054891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.056688 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.057027 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.057031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.057031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.057090 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.057307 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.057308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.057309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.057454 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.057617 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.057921 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.057987 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260619 12:09:52.060736 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.053441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45420)
0619 12:09:52.053489 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.053500 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.053538 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.053782 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.053783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.053934 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.053988 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.053990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.054690 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.054691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.055273 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.055275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.055797 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.055799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.055879 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.056262 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.056267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.057615 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.059843 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.059844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.059845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.060054 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.060224 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.060225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.060225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.060277 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.060465 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.060465 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.060496 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.060556 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.060626 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 12:09:52.060752 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.053518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45420 (local address 127.0.201.61:37357)
0619 12:09:52.053652 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.053654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.053659 (+ 5us) server_negotiation.cc:407] Connection header received
0619 12:09:52.053671 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.053672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.053679 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.053778 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.054054 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.054588 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.055378 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.055720 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.055783 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.057676 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.057677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.057677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.057683 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.059752 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.060118 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.060118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.060119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.060164 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.060328 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.060328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.060328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.060415 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.060457 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.060571 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.060666 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260619 12:09:52.062011 7452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Bootstrap starting.
I20260619 12:09:52.062592 7452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:52.062947 7452 log.cc:824] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:52.063699 7452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: No bootstrap required, opened a new log
I20260619 12:09:52.065007 7452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } }
I20260619 12:09:52.065120 7452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:52.065141 7452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404e21e3716344df971cf67bcbb55655, State: Initialized, Role: FOLLOWER
I20260619 12:09:52.065265 7452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } }
I20260619 12:09:52.065562 7452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:52.065548 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } } }
I20260619 12:09:52.065670 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:09:52.067077 7470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:52.067147 7470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:52.068851 7384 master_runner.cc:428] Detected that this master 404e21e3716344df971cf67bcbb55655 is joining an existing cluster
I20260619 12:09:52.068902 7384 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.201.61:37357
I20260619 12:09:52.076855 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.070017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.0.201.61:37357)
0619 12:09:52.070047 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.070049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.070057 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.070122 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.070122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.070129 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.070172 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.070541 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.071037 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.072113 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.072439 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.072515 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.073449 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.073450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.073450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.073456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.075278 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.075681 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.075682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.075682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.075739 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.076195 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.076196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.076196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.076328 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.076398 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.076596 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.076734 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":15}
I20260619 12:09:52.077158 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.069841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45432)
0619 12:09:52.069998 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.070009 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.070062 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.070177 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.070178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.070357 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.070437 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.070439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.071124 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.071125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.072029 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.072031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.072542 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.072544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.072617 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.072993 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.072997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.073360 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.075370 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.075371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.075372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.075606 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.075810 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.075811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.075811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.075880 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.076407 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.076408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.076464 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.076564 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.076673 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"spinlock_wait_cycles":9600,"thread_start_us":118,"threads_started":1}
I20260619 12:09:52.078359 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.069894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.0.201.62:35305)
0619 12:09:52.069934 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.069936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.071399 (+ 1463us) server_negotiation.cc:407] Connection header received
0619 12:09:52.071475 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.071476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.071489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.071575 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.071940 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.072655 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.073605 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.073917 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.073998 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.075093 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.075094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.075094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.075102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.076834 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.077436 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.077438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.077438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.077504 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.077791 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.077791 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.077792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.077926 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.078024 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.078195 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.078269 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:52.078372 7477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.069842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:58672)
0619 12:09:52.071328 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.071338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.071410 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.071582 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.071583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.071785 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.071842 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.071844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.072739 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.072740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.073498 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.073500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.073998 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.073999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.074080 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.074643 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.074647 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.074980 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.077065 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.077066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.077067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.077331 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.077589 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.077590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.077590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.077652 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.078013 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.078014 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.078054 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.078157 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.078263 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.62@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
I20260619 12:09:52.088153 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.080680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.0.201.61:37357)
0619 12:09:52.080716 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.080718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.080872 (+ 154us) server_negotiation.cc:407] Connection header received
0619 12:09:52.080963 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.080963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.080972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.081016 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.081478 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.081931 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.083014 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.083262 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.083329 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.085397 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.085398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.085399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.085406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.086808 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.087241 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.087242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.087242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.087301 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.087523 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.087523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.087524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.087646 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.087728 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.087974 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.088047 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:52.088164 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.080629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45438)
0619 12:09:52.080770 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.080817 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.080892 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.081029 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.081032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.081349 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.081402 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.081404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.082012 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.082013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.082931 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.082933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.083344 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.083345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.083421 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.083868 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.083873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.085285 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.086912 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.086920 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.086922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.087129 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.087369 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.087371 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.087371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.087433 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.087722 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.087728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.087780 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.087924 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.088049 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":50,"threads_started":1}
I20260619 12:09:52.088752 6920 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.201.61:37357: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "404e21e3716344df971cf67bcbb55655"
member_type: NON_VOTER
last_known_addr {
host: "127.0.201.61"
port: 37357
}
attrs {
promote: true
}
}
dest_uuid: "1a4785b9c63e4fdba05b1257ea44c8af"
cas_config_opid_index: -1
I20260619 12:09:52.089059 6920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } }
I20260619 12:09:52.089497 7479 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 404e21e3716344df971cf67bcbb55655 (127.0.201.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } }
I20260619 12:09:52.089751 7426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:52.090454 7479 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.201.61:37357
I20260619 12:09:52.090517 7426 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Refusing update from remote peer 1a4785b9c63e4fdba05b1257ea44c8af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 12:09:52.090664 7384 master.cc:537] Master@127.0.201.61:37357 shutting down...
I20260619 12:09:52.090674 7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } }
I20260619 12:09:52.090746 7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.090991 7480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:09:52.091677 6904 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 12:09:52.093166 7384 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:52.093250 7384 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:52.093300 7384 tablet_replica.cc:330] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: stopping tablet replica
I20260619 12:09:52.095523 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.095423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45444)
0619 12:09:52.095467 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.095502 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:37357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:09:52.095631 6904 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af -> Peer 404e21e3716344df971cf67bcbb55655 (127.0.201.61:37357): Couldn't send request to peer 404e21e3716344df971cf67bcbb55655. Status: Network error: Client connection negotiation failed: client connection to 127.0.201.61:37357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:09:52.105767 7384 master.cc:552] Master@127.0.201.61:37357 shutdown complete.
I20260619 12:09:52.105827 7384 master_runner.cc:305] Clearing existing system tablet
I20260619 12:09:52.105955 7384 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.106000 7384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.106695 7384 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.107232 7484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.107352 7384 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.107414 7384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.107453 7384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
uuid: "404e21e3716344df971cf67bcbb55655"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "dc92c600339ce329037931d48153cef6"
server_key_iv: "2cbc412d18613e0d0cb4af267a1e4b60"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.108016 7384 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:52.109397 7384 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 12:09:52.109448 7384 log.cc:1196] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 12:09:52.109639 7384 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Deleting consensus metadata
I20260619 12:09:52.109802 7384 master_runner.cc:315] Copying system tablet from 127.0.201.62:35305
I20260619 12:09:52.110371 7384 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: tablet copy: Beginning tablet copy session from remote peer at address 127.0.201.62:35305
I20260619 12:09:52.116698 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.110684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.0.201.62:35305)
0619 12:09:52.110720 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.110722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.110814 (+ 92us) server_negotiation.cc:407] Connection header received
0619 12:09:52.110883 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.110884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.110892 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.110935 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.111316 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.111821 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.112867 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.113039 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.113089 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.114114 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.114115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.114115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.114121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.115611 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.115981 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.115982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.115982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.116028 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.116236 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.116237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.116237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.116330 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.116381 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.116557 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.116640 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:09:52.116710 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.110635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:58678)
0619 12:09:52.110756 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.110765 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.110827 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.110949 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.110950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.111143 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.111247 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.111249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.111898 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.111899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.112776 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.112779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.113108 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.113109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.113177 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.113597 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.113602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.113977 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.115685 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.115686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.115687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.115912 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.116091 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.116092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.116092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.116155 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.116393 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.116393 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.116437 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.116514 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.116606 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260619 12:09:52.117194 6940 tablet_copy_service.cc:140] P 1a4785b9c63e4fdba05b1257ea44c8af: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 404e21e3716344df971cf67bcbb55655 ({username='oryx', principal='oryx/127.0.201.61@KRBTEST.COM'} at 127.0.0.1:58678)
I20260619 12:09:52.117295 6940 tablet_copy_service.cc:161] P 1a4785b9c63e4fdba05b1257ea44c8af: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 404e21e3716344df971cf67bcbb55655 at {username='oryx', principal='oryx/127.0.201.61@KRBTEST.COM'} at 127.0.0.1:58678: session id = 404e21e3716344df971cf67bcbb55655-00000000000000000000000000000000
I20260619 12:09:52.118072 6940 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:09:52.118607 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 12:09:52.119516 7384 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: tablet copy: Starting download of 0 data blocks...
I20260619 12:09:52.119666 7384 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: tablet copy: Starting download of 1 WAL segments...
I20260619 12:09:52.120529 7384 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:09:52.121265 6940 tablet_copy_service.cc:342] P 1a4785b9c63e4fdba05b1257ea44c8af: Request end of tablet copy session 404e21e3716344df971cf67bcbb55655-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.201.61@KRBTEST.COM'} at 127.0.0.1:58678
I20260619 12:09:52.121328 6940 tablet_copy_service.cc:433] P 1a4785b9c63e4fdba05b1257ea44c8af: ending tablet copy session 404e21e3716344df971cf67bcbb55655-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 404e21e3716344df971cf67bcbb55655
I20260619 12:09:52.123539 7384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.124698 7491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.124811 7494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.124922 7384 server_base.cc:1034] running on GCE node
W20260619 12:09:52.125061 7492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.125219 7384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.125380 7384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.126516 7384 hybrid_clock.cc:648] HybridClock initialized: now 1781870992126507 us; error 24 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.127797 7384 init.cc:377] Logged in from keytab as oryx/127.0.201.61@KRBTEST.COM (short username oryx)
I20260619 12:09:52.128088 7384 webserver.cc:460] Webserver started at http://127.0.201.61:40267/ using document root <none> and password file <none>
I20260619 12:09:52.128242 7384 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.128284 7384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.128875 7384 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:52.129395 7501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.129509 7384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:52.129555 7384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
uuid: "404e21e3716344df971cf67bcbb55655"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "dc92c600339ce329037931d48153cef6"
server_key_iv: "2cbc412d18613e0d0cb4af267a1e4b60"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.129598 7384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.143805 7384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.145347 7384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.148341 7384 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.61:37357
I20260619 12:09:52.148511 7553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.61:37357 every 8 connection(s)
I20260619 12:09:52.148697 7384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
I20260619 12:09:52.149325 7554 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:09:52.149735 7554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Bootstrap starting.
I20260619 12:09:52.152002 7554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:09:52.152232 7554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: Bootstrap complete.
I20260619 12:09:52.152426 7554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } }
I20260619 12:09:52.152491 7554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 404e21e3716344df971cf67bcbb55655, State: Initialized, Role: LEARNER
I20260619 12:09:52.152515 7554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } }
I20260619 12:09:52.152671 7554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:52.152689 7557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } }
I20260619 12:09:52.152835 7557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:09:52.154075 7568 catalog_manager.cc:1259] Loaded cluster ID: ff452963a7924aa88c4c69f334abe857
I20260619 12:09:52.154122 7568 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: loading cluster ID for follower catalog manager: success
I20260619 12:09:52.155079 7568 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: acquiring CA information for follower catalog manager: success
I20260619 12:09:52.155390 7568 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:09:52.166855 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.159758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.0.201.61:37357)
0619 12:09:52.159880 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.159882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.159893 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:52.159919 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.159920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.159931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.160019 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.160625 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.161601 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.162313 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.162613 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.162711 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.163934 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.163936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.163936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.163944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.165619 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.165976 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.165977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.165978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.166048 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.166247 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.166248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.166248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.166386 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.166503 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.166646 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.166741 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.0.201.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.60.keytab.
Entry for principal oryx/127.0.201.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.60.keytab.
I20260619 12:09:52.190606 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:46105
--webserver_interface=127.0.201.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.60.keytab
--principal=oryx/127.0.201.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:52.272008 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:52.272243 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:52.272282 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:52.273985 7572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:52.274050 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:52.274080 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:52.274112 7572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:52.274142 7572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:52.275714 7572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.60.keytab
--principal=oryx/127.0.201.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:46105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.201.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:52.275995 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:52.276263 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.281407 7580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.281442 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.281437 7577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.281713 7572 server_base.cc:1034] running on GCE node
I20260619 12:09:52.281960 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.282263 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.283416 7572 hybrid_clock.cc:648] HybridClock initialized: now 1781870992283400 us; error 48 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.285840 7572 init.cc:377] Logged in from keytab as oryx/127.0.201.60@KRBTEST.COM (short username oryx)
I20260619 12:09:52.286314 7572 webserver.cc:460] Webserver started at http://127.0.201.60:39995/ using document root <none> and password file <none>
I20260619 12:09:52.286530 7572 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.286597 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.286753 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:52.287820 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/instance:
uuid: "3ff91b1f4a204e76887ea38e5512aa96"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "3490b9d4ef3ae4256879fe3808d44c78"
server_key_iv: "0149fddcb02efb75ad3c3b1bbc0c2957"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.288157 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal/instance:
uuid: "3ff91b1f4a204e76887ea38e5512aa96"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "3490b9d4ef3ae4256879fe3808d44c78"
server_key_iv: "0149fddcb02efb75ad3c3b1bbc0c2957"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.289371 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:09:52.290019 7587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.290184 7572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:52.290279 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
uuid: "3ff91b1f4a204e76887ea38e5512aa96"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "3490b9d4ef3ae4256879fe3808d44c78"
server_key_iv: "0149fddcb02efb75ad3c3b1bbc0c2957"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.290357 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.300765 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.302904 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:52.303092 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.307672 7572 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.60:46105
I20260619 12:09:52.307701 7639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.60:46105 every 8 connection(s)
I20260619 12:09:52.308020 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
I20260619 12:09:52.308880 7640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:52.310245 7640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:09:52.315107 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 7572
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.60@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:52.319308 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.310974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.0.201.62:35305)
0619 12:09:52.311038 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.311040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.311118 (+ 78us) server_negotiation.cc:407] Connection header received
0619 12:09:52.311641 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.311642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.311651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.311710 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.312248 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.312721 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.313719 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.313905 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.313953 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.316204 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.316206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.316206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.316215 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.317730 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.318217 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.318218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.318219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.318294 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.318576 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.318577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.318577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.318698 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.318805 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.319081 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.319170 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:09:52.319471 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.310851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:58682)
0619 12:09:52.310998 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.311036 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.311560 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.311788 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.311791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.312009 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.312180 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.312183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.312827 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.312828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.313644 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.313645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.313974 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.313975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.314075 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.314524 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.314532 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.316122 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.317805 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.317811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.317813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.318100 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.318380 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.318381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.318381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.318507 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.318810 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.318817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.318944 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.319058 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.319203 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260619 12:09:52.319960 7640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.201.60@KRBTEST.COM
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.60@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
I20260619 12:09:52.324674 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.315664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.0.201.60:46105)
0619 12:09:52.315816 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.315899 (+ 83us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.315920 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:09:52.315947 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.315949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.315966 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.316051 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.316801 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.317626 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.318597 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.318888 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.318974 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.321749 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.321755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.321756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.321763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.323465 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.323824 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.323826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.323827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.323887 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.324124 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.324125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.324125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.324263 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.324402 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.324492 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.324564 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260619 12:09:52.327000 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.320217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45480)
0619 12:09:52.320271 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.320284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.320333 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.320458 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.320459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.320605 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.320661 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.320663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.321851 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.321852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.322593 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.322596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.323142 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.323142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.323230 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.323596 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.323601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.324809 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.326259 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.326260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.326261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.326410 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.326581 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.326581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.326581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.326627 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.326797 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.326798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.326829 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.326883 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.326945 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:09:52.327029 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.320272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.0.201.61:37357)
0619 12:09:52.320314 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.320316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.320325 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:52.320397 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.320398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.320409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.320452 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.321291 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.321749 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.322700 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.323051 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.323129 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.324913 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.324915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.324915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.324923 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.326186 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.326474 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.326475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.326475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.326524 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.326668 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.326668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.326668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.326750 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.326792 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.326908 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.326970 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:09:52.327489 7640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.60@KRBTEST.COM for oryx/127.0.201.60@KRBTEST.COM
I20260619 12:09:52.333410 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.327714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.0.1:45248)
0619 12:09:52.327760 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.327772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.327817 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.327937 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.327938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.328065 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.328086 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.328087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.328757 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.328758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.329213 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.329215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.329620 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.329621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.329667 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.330065 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.330070 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.331227 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.332465 (+ 1238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.332466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.332466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.332619 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.332825 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.332826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.332827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.332878 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.333114 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.333115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.333149 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.333211 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.333304 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:09:52.333458 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.327779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.0.201.60:46105)
0619 12:09:52.327806 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.327807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.327815 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.327872 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.327873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.327882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.327935 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.328167 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.328670 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.329313 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.329555 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.329618 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.331304 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.331306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.331306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.331312 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.332371 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.332702 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.332702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.332703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.332758 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.332948 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.332948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.332949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.333051 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.333123 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.333228 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.333352 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":13}
I20260619 12:09:52.334722 7640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Bootstrap starting.
I20260619 12:09:52.335577 7640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:52.335843 7640 log.cc:824] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:52.336501 7640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: No bootstrap required, opened a new log
I20260619 12:09:52.337942 7640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } }
I20260619 12:09:52.338060 7640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:52.338106 7640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff91b1f4a204e76887ea38e5512aa96, State: Initialized, Role: FOLLOWER
I20260619 12:09:52.338253 7640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } }
I20260619 12:09:52.338533 7640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:52.338510 7645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } } }
I20260619 12:09:52.338608 7645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:09:52.339985 7656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:09:52.340075 7656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:09:52.342554 7572 master_runner.cc:428] Detected that this master 3ff91b1f4a204e76887ea38e5512aa96 is joining an existing cluster
I20260619 12:09:52.342618 7572 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.201.60:46105
I20260619 12:09:52.350306 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.344185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.0.201.61:37357)
0619 12:09:52.344247 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.344249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.344561 (+ 312us) server_negotiation.cc:407] Connection header received
0619 12:09:52.344638 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.344639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.344648 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.344689 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.345033 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.345501 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.346572 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.346739 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.346788 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.347839 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.347840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.347840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.347847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.349222 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.349612 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.349612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.349612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.349660 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.349840 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.349840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.349841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.349933 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.349982 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.350173 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.350235 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:09:52.350361 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.344086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45490)
0619 12:09:52.344506 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.344517 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.344580 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.344702 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.344703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.344893 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.344967 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.344969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.345578 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.345579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.346494 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.346496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.346815 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.346816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.346874 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.347368 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.347373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.347761 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.349312 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.349313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.349314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.349537 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.349719 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.349719 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.349720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.349781 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.349995 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.349996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.350039 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.350145 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.350244 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":88,"threads_started":1}
I20260619 12:09:52.356798 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.344086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:58694)
0619 12:09:52.350417 (+ 6331us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.350427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.350474 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.350611 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.350612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.350752 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.350801 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.350803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.351780 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.351780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.352693 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.352695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.353141 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.353142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.353209 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.353531 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.353535 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.353863 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.355640 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.355641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.355641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.355869 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.356136 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.356136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.356137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.356192 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.356480 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.356481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.356548 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.356614 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.356700 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6322,"mutex_wait_us":28,"spinlock_wait_cycles":9984,"thread_start_us":131,"threads_started":1}
I20260619 12:09:52.356837 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.344177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.0.201.62:35305)
0619 12:09:52.344238 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.344240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.350473 (+ 6233us) server_negotiation.cc:407] Connection header received
0619 12:09:52.350530 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.350531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.350545 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.350597 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.350939 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.351684 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.352787 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.353019 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.353140 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.353954 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.353955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.353955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.353961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.355576 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.355964 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.355965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.355966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.356022 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.356262 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.356263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.356264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.356393 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.356435 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.356666 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.356743 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870991, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.62@KRBTEST.COM for oryx/127.0.201.60@KRBTEST.COM
I20260619 12:09:52.375968 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6982
I20260619 12:09:52.378773 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.344361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.0.1:45252)
0619 12:09:52.355334 (+ 10973us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.355345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.355411 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.355554 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.355555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.355759 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.355805 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.355806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.356435 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.356436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.357562 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.357577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.358165 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.358166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.358263 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.375713 (+ 17450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.375718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.376116 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.377849 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.377850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.377850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.378035 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.378195 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.378195 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.378196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.378248 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.378495 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.378495 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.378526 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.378583 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.378664 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10944,"thread_start_us":77,"threads_started":1}
I20260619 12:09:52.379022 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.344426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.0.201.60:46105)
0619 12:09:52.344458 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.344460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.355414 (+ 10954us) server_negotiation.cc:407] Connection header received
0619 12:09:52.355465 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.355466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.355476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.355524 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.355888 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.356360 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.357653 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.357940 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.358003 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.376203 (+ 18200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.376204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.376204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.376209 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.377770 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.378100 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.378100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.378101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.378140 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.378301 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.378302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.378302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.378425 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.378476 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.378864 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.378949 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:09:52.381201 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.358801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45256 (local address 127.0.201.60:46105)
0619 12:09:52.358951 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.358953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.358959 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:09:52.358986 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.358986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.358993 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.359046 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.359349 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.359793 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.360728 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.360914 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.360967 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.362742 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.362743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.362744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.362758 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.379940 (+ 17182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.380368 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.380369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.380369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.380432 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.380660 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.380661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.380661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.380788 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.380853 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.381027 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.381102 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":106,"threads_started":1}
I20260619 12:09:52.381271 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.358723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.0.1:45256)
0619 12:09:52.358764 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.358773 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.358804 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.359068 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.359069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.359244 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.359283 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.359285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.359861 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.359862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.360656 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.360658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.360987 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.360988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.361051 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.361370 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.361375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.362657 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.380036 (+ 17379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.380038 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.380038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.380290 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.380524 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.380525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.380525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.380594 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.380863 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.380864 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.380914 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.380991 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.381109 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 12:09:52.381770 6920 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.201.60:46105: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96"
member_type: NON_VOTER
last_known_addr {
host: "127.0.201.60"
port: 46105
}
attrs {
promote: true
}
}
dest_uuid: "1a4785b9c63e4fdba05b1257ea44c8af"
cas_config_opid_index: 5
I20260619 12:09:52.381928 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:33203
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:35305
--webserver_port=43101
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:09:52.382033 6920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } }
I20260619 12:09:52.382443 7479 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.382766 7614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:52.383508 7479 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.201.60:46105
I20260619 12:09:52.383594 7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.383636 7614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Refusing update from remote peer 1a4785b9c63e4fdba05b1257ea44c8af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 12:09:52.383672 7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.383981 7572 master.cc:537] Master@127.0.201.60:46105 shutting down...
I20260619 12:09:52.384182 7480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:09:52.384913 6904 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af -> Peer 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60:46105): Couldn't send request to peer 3ff91b1f4a204e76887ea38e5512aa96. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 12:09:52.386942 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.382476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45492)
0619 12:09:52.383336 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.383347 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.383400 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.383615 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.383616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.383618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.383656 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.383657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.384540 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.384541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.385778 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.385781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.386664 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.386664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.386730 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.386748 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.386801 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.386842 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842}
I20260619 12:09:52.386953 7572 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:52.387030 7572 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:52.387076 7572 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: stopping tablet replica
I20260619 12:09:52.387225 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.383276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.0.201.61:37357)
0619 12:09:52.383328 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.383330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.383387 (+ 57us) server_negotiation.cc:407] Connection header received
0619 12:09:52.383456 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.383457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.383480 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.383549 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.383731 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.384447 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.385874 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.386481 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.386550 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.386628 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.387100 (+ 472us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.387135 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:09:52.387423 7528 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 LEARNER]: Refusing update from remote peer 1a4785b9c63e4fdba05b1257ea44c8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 12:09:52.387609 7480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 12:09:52.388193 7528 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.389104 7557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a4785b9c63e4fdba05b1257ea44c8af. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.389189 7557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:09:52.389230 7557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: NON_VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: true } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.389281 7557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:09:52.389290 7479 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: attempting to promote NON_VOTER 404e21e3716344df971cf67bcbb55655 to VOTER
I20260619 12:09:52.389457 7479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } }
I20260619 12:09:52.389912 7528 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 LEARNER]: Refusing update from remote peer 1a4785b9c63e4fdba05b1257ea44c8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 12:09:52.390086 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.389998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.0.1:45266)
0619 12:09:52.390033 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.390059 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:46105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 12:09:52.390123 7480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260619 12:09:52.390179 6904 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af -> Peer 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60:46105): Couldn't send request to peer 3ff91b1f4a204e76887ea38e5512aa96. Status: Network error: Client connection negotiation failed: client connection to 127.0.201.60:46105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:09:52.390925 7480 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 404e21e3716344df971cf67bcbb55655 (127.0.201.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.391851 7480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.391958 7480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.392017 7480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.392068 7480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.392185 7528 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 404e21e3716344df971cf67bcbb55655 (127.0.201.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.392836 7668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.392979 7668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:52.402076 7572 master.cc:552] Master@127.0.201.60:46105 shutdown complete.
I20260619 12:09:52.402137 7572 master_runner.cc:305] Clearing existing system tablet
I20260619 12:09:52.402258 7572 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.402312 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.402876 7572 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260619 12:09:52.403349 7671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.403492 7572 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.403541 7572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.403568 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
uuid: "3ff91b1f4a204e76887ea38e5512aa96"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "3490b9d4ef3ae4256879fe3808d44c78"
server_key_iv: "0149fddcb02efb75ad3c3b1bbc0c2957"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.404227 7572 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:52.405575 7572 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 12:09:52.405634 7572 log.cc:1196] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 12:09:52.405834 7572 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Deleting consensus metadata
I20260619 12:09:52.405963 7572 master_runner.cc:315] Copying system tablet from 127.0.201.62:35305
I20260619 12:09:52.406584 7572 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: tablet copy: Beginning tablet copy session from remote peer at address 127.0.201.62:35305
I20260619 12:09:52.412927 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.407049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.0.201.62:35305)
0619 12:09:52.407087 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.407089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.407097 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.407116 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.407117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.407125 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.407180 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.407498 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.408002 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.408734 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.408940 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.409024 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.410007 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.410009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.410009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.410016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.411708 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.412102 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.412103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.412104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.412163 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.412367 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.412368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.412368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.412503 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.412545 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.412757 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.412837 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:52.413240 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.406871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:58706)
0619 12:09:52.406989 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.407001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.407053 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.407185 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.407186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.407372 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.407412 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.407414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.408080 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.408081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.408632 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.408634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.409025 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.409026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.409083 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.409596 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.409601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.409882 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.411799 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.411800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.411800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.412009 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.412236 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.412237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.412238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.412295 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.412591 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.412592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.412649 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.413033 (+ 384us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.413136 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":85,"threads_started":1}
I20260619 12:09:52.413465 6940 tablet_copy_service.cc:140] P 1a4785b9c63e4fdba05b1257ea44c8af: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3ff91b1f4a204e76887ea38e5512aa96 ({username='oryx', principal='oryx/127.0.201.60@KRBTEST.COM'} at 127.0.0.1:58706)
I20260619 12:09:52.413511 6940 tablet_copy_service.cc:161] P 1a4785b9c63e4fdba05b1257ea44c8af: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3ff91b1f4a204e76887ea38e5512aa96 at {username='oryx', principal='oryx/127.0.201.60@KRBTEST.COM'} at 127.0.0.1:58706: session id = 3ff91b1f4a204e76887ea38e5512aa96-00000000000000000000000000000000
I20260619 12:09:52.414016 6940 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:09:52.414500 7572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 12:09:52.415526 7572 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: tablet copy: Starting download of 0 data blocks...
I20260619 12:09:52.415674 7572 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: tablet copy: Starting download of 1 WAL segments...
I20260619 12:09:52.416325 7572 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:09:52.417006 6940 tablet_copy_service.cc:342] P 1a4785b9c63e4fdba05b1257ea44c8af: Request end of tablet copy session 3ff91b1f4a204e76887ea38e5512aa96-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.201.60@KRBTEST.COM'} at 127.0.0.1:58706
I20260619 12:09:52.417078 6940 tablet_copy_service.cc:433] P 1a4785b9c63e4fdba05b1257ea44c8af: ending tablet copy session 3ff91b1f4a204e76887ea38e5512aa96-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3ff91b1f4a204e76887ea38e5512aa96
I20260619 12:09:52.419382 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.420517 7678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.420532 7681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.420719 7572 server_base.cc:1034] running on GCE node
W20260619 12:09:52.423331 7679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.423472 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.423619 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.424741 7572 hybrid_clock.cc:648] HybridClock initialized: now 1781870992424730 us; error 28 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.426025 7572 init.cc:377] Logged in from keytab as oryx/127.0.201.60@KRBTEST.COM (short username oryx)
I20260619 12:09:52.426295 7572 webserver.cc:460] Webserver started at http://127.0.201.60:37025/ using document root <none> and password file <none>
I20260619 12:09:52.426424 7572 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.426460 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.427084 7572 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:52.427618 7688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.427763 7572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.427811 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
uuid: "3ff91b1f4a204e76887ea38e5512aa96"
format_stamp: "Formatted at 2026-06-19 12:09:52 on dist-test-slave-c910"
server_key: "3490b9d4ef3ae4256879fe3808d44c78"
server_key_iv: "0149fddcb02efb75ad3c3b1bbc0c2957"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.427868 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.442330 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.444087 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.447063 7572 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.60:46105
I20260619 12:09:52.447435 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
I20260619 12:09:52.448134 7741 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:09:52.448544 7741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Bootstrap starting.
I20260619 12:09:52.450644 7741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:09:52.450871 7741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Bootstrap complete.
I20260619 12:09:52.451076 7741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } }
I20260619 12:09:52.451185 7741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3ff91b1f4a204e76887ea38e5512aa96, State: Initialized, Role: LEARNER
I20260619 12:09:52.451243 7741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } }
I20260619 12:09:52.451406 7741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:52.451382 7744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.451583 7744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:09:52.452787 7755 catalog_manager.cc:1259] Loaded cluster ID: ff452963a7924aa88c4c69f334abe857
I20260619 12:09:52.452880 7755 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: loading cluster ID for follower catalog manager: success
I20260619 12:09:52.454020 7755 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: acquiring CA information for follower catalog manager: success
I20260619 12:09:52.454371 7755 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:09:52.459312 7740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.60:46105 every 8 connection(s)
W20260619 12:09:52.498790 7666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:52.499013 7666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:52.499053 7666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:52.500236 7666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:52.500300 7666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:52.500345 7666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:52.501824 7666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:33203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=43101
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:52.502086 7666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:52.502312 7666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.507651 7761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.507745 7760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.507663 7763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.508018 7666 server_base.cc:1034] running on GCE node
I20260619 12:09:52.508225 7666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.508476 7666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.509649 7666 hybrid_clock.cc:648] HybridClock initialized: now 1781870992509635 us; error 27 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.512974 7666 init.cc:377] Logged in from keytab as oryx/127.0.201.1@KRBTEST.COM (short username oryx)
I20260619 12:09:52.513428 7666 webserver.cc:460] Webserver started at http://127.0.201.1:43101/ using document root <none> and password file <none>
I20260619 12:09:52.513648 7666 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.513725 7666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.515053 7666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:52.515725 7770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.515904 7666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:52.515997 7666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "5b3ed0457e54445c8ddf7e804eee797c"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "24c7043db5fc309be298192e37a9b088"
server_key_iv: "f281cd82145bd3d3d1cdf7de3e453668"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.516335 7666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.532045 7666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.534350 7666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:52.534529 7666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.534758 7666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:52.535051 7666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:52.535104 7666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.535143 7666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:52.535171 7666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.541323 7666 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:33203
I20260619 12:09:52.541417 7883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:33203 every 8 connection(s)
I20260619 12:09:52.541736 7666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:09:52.547878 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 7666
I20260619 12:09:52.548064 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 7116
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.60@KRBTEST.COM
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:52.553349 7890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.543128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.201.1:39885)
0619 12:09:52.543267 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.543277 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.543692 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.544210 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.544211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.544457 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.544642 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.544645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.545489 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.545490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.546624 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.546625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.547112 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.547113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.547175 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.547651 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.547656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.549520 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.551671 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.551678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.551680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.552013 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.552216 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.552217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.552217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.552356 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.552641 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.552648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.552807 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.552940 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.553112 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":63,"thread_start_us":53,"threads_started":1}
I20260619 12:09:52.553646 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.542840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:37051 (local address 127.0.201.62:35305)
0619 12:09:52.542880 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.542882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.543157 (+ 275us) server_negotiation.cc:407] Connection header received
0619 12:09:52.543795 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.543796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.543809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.543841 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.544747 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.545485 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.546702 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.546993 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.547086 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.550368 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.550369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.550369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.550377 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.552428 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.552781 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.552782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.552783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.552841 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.553022 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.553023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.553024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.553162 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.553229 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.553421 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.553505 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:09:52.553864 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.543269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:39885 (local address 127.0.201.60:46105)
0619 12:09:52.543390 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.543399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.543408 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:52.543775 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.543776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.543790 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.543859 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.544736 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.545378 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.546733 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.546999 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.547061 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.549616 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.549617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.549617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.549625 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.551577 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.552098 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.552099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.552100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.552159 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.552434 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.552435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.552435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.552582 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.552653 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.553675 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.553774 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260619 12:09:52.553648 7889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.542850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:37051)
0619 12:09:52.543026 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.543072 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.543674 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.543945 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.543949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.544435 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.544642 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.544647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.545802 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.545802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.546623 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.546625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.547089 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.547089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.547155 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.548922 (+ 1767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.548926 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.550277 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.552516 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.552517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.552517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.552711 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.552910 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.552911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.552912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.552972 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.553232 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.553233 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.553271 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.553437 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.553549 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":33,"spinlock_wait_cycles":768,"thread_start_us":99,"threads_started":1}
I20260619 12:09:52.554481 7885 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35305
I20260619 12:09:52.554582 7885 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.554774 7885 heartbeater.cc:511] Master 127.0.201.62:35305 requested a full tablet report, sending...
I20260619 12:09:52.555117 7884 heartbeater.cc:347] Connected to a master server at 127.0.201.60:46105
I20260619 12:09:52.555166 7884 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.555217 6920 ts_manager.cc:194] Re-registered known tserver with Master: 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203)
I20260619 12:09:52.555240 7884 heartbeater.cc:511] Master 127.0.201.60:46105 requested a full tablet report, sending...
I20260619 12:09:52.555601 7705 ts_manager.cc:194] Registered new tserver with Master: 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203)
I20260619 12:09:52.555707 6920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:37051
I20260619 12:09:52.555858 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:44847
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:35305
--webserver_port=44963
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
I20260619 12:09:52.559254 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.542850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.201.1:55547)
0619 12:09:52.543026 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.543072 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.543674 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.543957 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.543958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.544459 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.544642 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.544647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.545570 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.545570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.546457 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.546459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.546914 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.546915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.547062 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.547540 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.547548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.556568 (+ 9020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.558156 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.558157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.558158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.558323 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.558542 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.558543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.558543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.558597 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.558886 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.558887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.558949 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.559018 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.559129 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":60,"thread_start_us":88,"threads_started":1}
I20260619 12:09:52.559254 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.543093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:55547 (local address 127.0.201.61:37357)
0619 12:09:52.543141 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.543143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.543155 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:09:52.543830 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.543832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.543843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.543890 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.544748 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.545491 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.546564 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.546838 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.546907 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.556655 (+ 9748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.556657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.556657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.556665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.558059 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.558414 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.558415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.558416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.558470 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.558666 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.558667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.558667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.558778 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.558860 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.559057 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.559129 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:09:52.559948 7886 heartbeater.cc:347] Connected to a master server at 127.0.201.61:37357
I20260619 12:09:52.559999 7886 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.560071 7886 heartbeater.cc:511] Master 127.0.201.61:37357 requested a full tablet report, sending...
I20260619 12:09:52.560434 7518 ts_manager.cc:194] Registered new tserver with Master: 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203)
W20260619 12:09:52.635061 7892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:52.635342 7892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:52.635408 7892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:52.636543 7892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:52.636579 7892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:52.636623 7892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:52.638109 7892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:44847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=44963
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:52.638278 7892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:52.638485 7892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.643358 7898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.643383 7892 server_base.cc:1034] running on GCE node
W20260619 12:09:52.643414 7900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.643358 7897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.643810 7892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.644045 7892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.645205 7892 hybrid_clock.cc:648] HybridClock initialized: now 1781870992645198 us; error 46 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.647712 7892 init.cc:377] Logged in from keytab as oryx/127.0.201.2@KRBTEST.COM (short username oryx)
I20260619 12:09:52.648123 7892 webserver.cc:460] Webserver started at http://127.0.201.2:44963/ using document root <none> and password file <none>
I20260619 12:09:52.648344 7892 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.648428 7892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.649664 7892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:52.650234 7907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.650404 7892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.650524 7892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "2c149b0fb6b94ede882ed006d1442da6"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "e5ef9a4824797abba64b6b01294c50e5"
server_key_iv: "8c59507ab0f95259aff6405dbfc4ed95"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.650941 7892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.675499 7892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.677196 7892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:52.677300 7892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.677469 7892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:52.677783 7892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:52.677817 7892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.677839 7892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:52.677851 7892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.683172 7892 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44847
I20260619 12:09:52.683280 8020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44847 every 8 connection(s)
I20260619 12:09:52.683619 7892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:52.690609 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 7892
I20260619 12:09:52.690762 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 7250
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.60@KRBTEST.COM
I20260619 12:09:52.695397 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.684779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:39915 (local address 127.0.201.61:37357)
0619 12:09:52.684843 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.684845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.684853 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.685745 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.685747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.685759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.685791 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.686579 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.687350 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688577 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.688863 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688903 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.692052 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.692054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.692054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.692062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.693634 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.694120 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.694121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.694121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.694180 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.694680 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.694680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.694681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.694782 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.694837 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.695174 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.695277 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:09:52.695631 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.684631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.201.2:39915)
0619 12:09:52.684768 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.684777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.685645 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.685883 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.685884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.686287 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.686477 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.686482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.687746 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.687747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.688498 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.688500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.688979 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.689045 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.689553 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.689558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.691956 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.693713 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.693719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.693721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.694035 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.694258 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.694259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.694259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.694392 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.694854 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.694860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.695030 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.695143 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.695344 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":26,"thread_start_us":56,"threads_started":1}
I20260619 12:09:52.696156 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.684309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.201.2:46177)
0619 12:09:52.684505 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.684546 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.685620 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.685870 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.685874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.686286 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.686475 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.686479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.687163 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.687164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.688290 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.688291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.688804 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.688881 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.689394 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.689399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.692583 (+ 3184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.694860 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.694861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.694861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.695008 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.695241 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.695242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.695242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.695297 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.695801 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.695802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.695841 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.695894 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.695991 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260619 12:09:52.696187 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.684297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:46177 (local address 127.0.201.60:46105)
0619 12:09:52.684400 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.684402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.684652 (+ 250us) server_negotiation.cc:407] Connection header received
0619 12:09:52.685731 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.685733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.685746 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.685802 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.686575 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.687063 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688384 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.688696 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688734 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.692674 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.692676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.692676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.692684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.694401 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.695084 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.695085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.695085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.695143 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.695368 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.695369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.695370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.695502 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.695550 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.695969 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.696055 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:09:52.696435 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:42477
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:35305
--webserver_port=41877
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:37345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:09:52.696373 8023 heartbeater.cc:347] Connected to a master server at 127.0.201.61:37357
I20260619 12:09:52.696626 8023 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.696782 8023 heartbeater.cc:511] Master 127.0.201.61:37357 requested a full tablet report, sending...
I20260619 12:09:52.697219 7518 ts_manager.cc:194] Registered new tserver with Master: 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847)
I20260619 12:09:52.697263 8021 heartbeater.cc:347] Connected to a master server at 127.0.201.60:46105
I20260619 12:09:52.697324 8021 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.697400 8021 heartbeater.cc:511] Master 127.0.201.60:46105 requested a full tablet report, sending...
I20260619 12:09:52.697683 7705 ts_manager.cc:194] Registered new tserver with Master: 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847)
I20260619 12:09:52.701612 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.684309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:54253)
0619 12:09:52.684507 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.684568 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.685645 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.685883 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.685884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.686314 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.686477 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.686482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.687426 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.687427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.688188 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.688190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.688599 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.688735 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.689220 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.689229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.697726 (+ 8497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.700267 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.700268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.700269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.700475 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.700705 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.700706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.700706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.700766 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.701186 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.701186 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.701261 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.701358 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.701481 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":49,"thread_start_us":110,"threads_started":1}
I20260619 12:09:52.701642 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.684315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54253 (local address 127.0.201.62:35305)
0619 12:09:52.684374 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.684377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.684674 (+ 297us) server_negotiation.cc:407] Connection header received
0619 12:09:52.685745 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.685747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.685759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.685791 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.686580 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.687346 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688278 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.688471 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.688592 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.697905 (+ 9313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.697906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.697907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.697915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.700172 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.700565 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.700566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.700566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.700637 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.700841 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.700842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.700842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.700973 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.701014 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.701414 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.701510 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:09:52.704517 8022 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35305
I20260619 12:09:52.704576 8022 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.704649 8022 heartbeater.cc:511] Master 127.0.201.62:35305 requested a full tablet report, sending...
I20260619 12:09:52.704910 6920 ts_manager.cc:194] Re-registered known tserver with Master: 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847)
I20260619 12:09:52.705652 6920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:54253
W20260619 12:09:52.784286 8028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:52.784482 8028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:52.784504 8028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:52.785653 8028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:52.785696 8028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:52.785723 8028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:52.787151 8028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:37345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:42477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=41877
--tserver_master_addrs=127.0.201.62:35305,127.0.201.61:37357,127.0.201.60:46105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:52.787434 8028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:52.787678 8028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:52.792986 8033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:52.792995 8034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.793152 8028 server_base.cc:1034] running on GCE node
W20260619 12:09:52.793018 8036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:52.793393 8028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:52.793619 8028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:52.794755 8028 hybrid_clock.cc:648] HybridClock initialized: now 1781870992794715 us; error 59 us; skew 500 ppm
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:52.797113 8028 init.cc:377] Logged in from keytab as oryx/127.0.201.3@KRBTEST.COM (short username oryx)
I20260619 12:09:52.797531 8028 webserver.cc:460] Webserver started at http://127.0.201.3:41877/ using document root <none> and password file <none>
I20260619 12:09:52.797780 8028 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:52.797858 8028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:52.799109 8028 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:52.799718 8043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.799873 8028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.799965 8028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "953edcf192924724ab15ed83aa6156bf"
format_stamp: "Formatted at 2026-06-19 12:09:51 on dist-test-slave-c910"
server_key: "4d8bdc0081e3ed71aedfb6cf9b1ef648"
server_key_iv: "4cad231b02c26df0eab8b65aed382c34"
server_key_version: "encryptionkey@0"
I20260619 12:09:52.800272 8028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:52.824234 8028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:52.826162 8028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:52.826313 8028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:52.826562 8028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:09:52.826836 8028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:52.826886 8028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.826922 8028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:52.826942 8028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:52.833539 8028 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:42477
I20260619 12:09:52.833583 8156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:42477 every 8 connection(s)
I20260619 12:09:52.833899 8028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.61@KRBTEST.COM
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:52.842104 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 8028
Jun 19 12:09:52 dist-test-slave-c910 krb5kdc[6875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870992, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.60@KRBTEST.COM
I20260619 12:09:52.845361 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.834964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:46641 (local address 127.0.201.61:37357)
0619 12:09:52.835016 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.835018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.835162 (+ 144us) server_negotiation.cc:407] Connection header received
0619 12:09:52.836109 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.836110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.836122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.836177 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.836959 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.837759 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.838810 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.839063 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839126 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.841602 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.841603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.841603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.841609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.843270 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.844175 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.844176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.844177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.844236 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.844510 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.844511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.844512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.844648 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.844708 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.845186 (+ 478us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.845260 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:09:52.845574 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.834883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.201.3:46641)
0619 12:09:52.835086 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.835095 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.835958 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.836232 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.836237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.836658 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.836851 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.836856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.837833 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.837834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.838722 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.838724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.839141 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.839294 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.839957 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.839966 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.841517 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.843803 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.843808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.843810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.844099 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.844296 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.844297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.844298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.844425 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.844732 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.844738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.844895 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.844999 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.845366 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":130,"thread_start_us":75,"threads_started":1}
I20260619 12:09:52.845816 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.834599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.201.3:50097)
0619 12:09:52.834777 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.834820 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.835942 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.836314 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.836316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.836632 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.836850 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.836854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.837679 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.837680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.838936 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.838938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.839476 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.839539 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.840310 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.840314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.842959 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.844810 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.844810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.844811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.845004 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.845189 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.845190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.845190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.845245 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.845514 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.845515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.845552 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.845616 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.845698 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":76,"thread_start_us":90,"threads_started":1}
I20260619 12:09:52.846256 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.834621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:50097 (local address 127.0.201.60:46105)
0619 12:09:52.834662 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.834664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.835040 (+ 376us) server_negotiation.cc:407] Connection header received
0619 12:09:52.836233 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.836236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.836249 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.836310 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.836944 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.837554 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839029 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.839350 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839424 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.843052 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.843053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.843053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.843060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.844739 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.845072 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.845073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.845073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.845124 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.845306 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.845307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.845307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.845428 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.845489 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.846067 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.846148 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:52.846511 8157 heartbeater.cc:347] Connected to a master server at 127.0.201.60:46105
I20260619 12:09:52.846585 8157 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.846740 8157 heartbeater.cc:511] Master 127.0.201.60:46105 requested a full tablet report, sending...
I20260619 12:09:52.847112 7705 ts_manager.cc:194] Registered new tserver with Master: 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477)
I20260619 12:09:52.847821 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.834591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:37859 (local address 127.0.201.62:35305)
0619 12:09:52.834640 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.834642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.835059 (+ 417us) server_negotiation.cc:407] Connection header received
0619 12:09:52.836061 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.836063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.836076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.836141 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.836959 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.837766 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.838996 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.839291 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839364 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.842715 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.842716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.842716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.842733 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.845003 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.846689 (+ 1686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.846690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.846690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.846772 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.847011 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.847012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.847012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.847129 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.847185 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.847586 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.847665 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:09:52.847829 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.834599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:37859)
0619 12:09:52.834995 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.835006 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.835942 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.836232 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.836237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.836658 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:52.836851 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.836856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.837962 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.837963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.838914 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.838916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.839362 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.839363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.839435 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.840108 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:52.840112 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:52.842643 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:52.845082 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.845083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.845083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.845285 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.846854 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:52.846855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:52.846855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:52.846916 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:52.847240 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:52.847241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:52.847281 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.847343 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.847434 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":44,"thread_start_us":88,"threads_started":1}
I20260619 12:09:52.848125 8158 heartbeater.cc:347] Connected to a master server at 127.0.201.62:35305
I20260619 12:09:52.848174 8158 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.848230 8158 heartbeater.cc:511] Master 127.0.201.62:35305 requested a full tablet report, sending...
I20260619 12:09:52.848424 6920 ts_manager.cc:194] Re-registered known tserver with Master: 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477)
I20260619 12:09:52.848910 6920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:37859
I20260619 12:09:52.847445 8159 heartbeater.cc:347] Connected to a master server at 127.0.201.61:37357
I20260619 12:09:52.849125 8159 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:52.849196 8159 heartbeater.cc:511] Master 127.0.201.61:37357 requested a full tablet report, sending...
I20260619 12:09:52.849476 7518 ts_manager.cc:194] Registered new tserver with Master: 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477)
I20260619 12:09:52.852003 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.843935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.0.201.60:46105)
0619 12:09:52.844241 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.844243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.844250 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:52.844268 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.844268 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.844281 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.844338 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.845084 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.845785 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.846882 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.847163 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.847258 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.848759 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.848760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.848760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.848766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.850765 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.851168 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.851169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.851169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.851251 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.851432 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.851433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.851434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.851543 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.851652 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.851808 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.851880 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":48,"threads_started":1}
I20260619 12:09:52.853497 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.844029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.0.201.61:37357)
0619 12:09:52.845432 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.845433 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.845440 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:52.845456 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.845457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.845466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.845513 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.846244 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.846880 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.848189 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.848453 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.848509 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.850171 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.850172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.850173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.850179 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.852317 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.852790 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.852791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.852792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.852840 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.853045 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.853046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.853046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.853145 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.853236 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.853331 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.853408 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1390,"thread_start_us":54,"threads_started":1}
I20260619 12:09:52.856030 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.843523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.0.201.62:35305)
0619 12:09:52.844060 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.844062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.849381 (+ 5319us) server_negotiation.cc:407] Connection header received
0619 12:09:52.849416 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.849417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.849430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.850103 (+ 673us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.850140 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.850901 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.851689 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.851996 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.852102 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.853533 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:52.853535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:52.853535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:52.853542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:52.854836 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.855246 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.855247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.855247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.855308 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:52.855526 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:52.855527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:52.855527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:52.855635 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:52.855734 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.855842 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.855929 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":67,"threads_started":1}
I20260619 12:09:52.857359 6920 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:52.858758 6920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:52.865888 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.862137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:42477 (local address 127.0.0.1:33036)
0619 12:09:52.862189 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.862200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.862238 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.862770 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.862772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.862774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.862817 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.862818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.863820 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.863821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.864772 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.864775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.865647 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.865648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.865714 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.865718 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.865780 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.865818 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":4}
I20260619 12:09:52.865911 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.862251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.0.201.3:42477)
0619 12:09:52.862582 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.862587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.862597 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:09:52.862637 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.862639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.862655 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.862766 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.862902 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.863729 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.864872 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.865569 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.865649 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.865728 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.865794 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.865830 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260619 12:09:52.866632 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.862435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.0.201.2:44847)
0619 12:09:52.862868 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.862872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.862880 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.862918 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.862920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.862934 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.863033 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.863152 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.864210 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.865338 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.866299 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.866393 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.866461 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.866523 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.866557 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":85,"threads_started":1}
I20260619 12:09:52.866642 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.862319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44847 (local address 127.0.0.1:60254)
0619 12:09:52.862543 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.862551 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.862585 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.863037 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.863037 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.863039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.863084 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.863085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.864308 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.864309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.865259 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.865261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.866382 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.866383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.866442 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.866446 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.866500 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.866543 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260619 12:09:52.866665 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.862144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33203 (local address 127.0.0.1:59210)
0619 12:09:52.862407 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.862418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.862464 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.862861 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.862862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.862864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.862929 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.862931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.864276 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.864277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.865382 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.865384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.866416 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.866417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.866484 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.866488 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.866519 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.866559 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":36,"thread_start_us":80,"threads_started":1}
I20260619 12:09:52.866716 8091 tablet_service.cc:1467] Processing CreateTablet for tablet db2c06a833ec4780bfdc07bed06045b8 (DEFAULT_TABLE table=test-table [id=f4c491c9eed24c759d6840a129dcb6e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:52.867102 8091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db2c06a833ec4780bfdc07bed06045b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:52.867269 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.862502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.0.201.1:33203)
0619 12:09:52.862702 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.862706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.862715 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:52.862751 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.862753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.862767 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.863004 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.863063 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.864177 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.865461 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.866327 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.867012 (+ 685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.867098 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.867122 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.867168 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260619 12:09:52.867621 7955 tablet_service.cc:1467] Processing CreateTablet for tablet db2c06a833ec4780bfdc07bed06045b8 (DEFAULT_TABLE table=test-table [id=f4c491c9eed24c759d6840a129dcb6e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:52.867792 7818 tablet_service.cc:1467] Processing CreateTablet for tablet db2c06a833ec4780bfdc07bed06045b8 (DEFAULT_TABLE table=test-table [id=f4c491c9eed24c759d6840a129dcb6e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:52.867951 7955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db2c06a833ec4780bfdc07bed06045b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:52.868125 7818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db2c06a833ec4780bfdc07bed06045b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:52.870599 8183 tablet_bootstrap.cc:492] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Bootstrap starting.
I20260619 12:09:52.871248 8183 tablet_bootstrap.cc:654] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:52.871260 8184 tablet_bootstrap.cc:492] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Bootstrap starting.
I20260619 12:09:52.871933 8184 tablet_bootstrap.cc:654] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:52.872143 8183 log.cc:824] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:52.872232 8184 log.cc:824] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:52.872454 8185 tablet_bootstrap.cc:492] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Bootstrap starting.
I20260619 12:09:52.872859 8183 tablet_bootstrap.cc:492] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: No bootstrap required, opened a new log
I20260619 12:09:52.872933 8183 ts_tablet_manager.cc:1397] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:09:52.874573 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.871634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.0.201.60:46105)
0619 12:09:52.871672 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.871673 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.871680 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:52.871702 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.871703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.871722 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.871795 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.871924 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.872645 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.873894 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.874257 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.874315 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.874349 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.874496 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.874528 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:52.874589 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.871540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.0.1:45282)
0619 12:09:52.871570 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.871581 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.871616 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.871808 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.871809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.871811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.871839 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.871841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.872729 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.872730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.873800 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.873802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.874333 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.874334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.874409 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.874413 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.874487 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.874525 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260619 12:09:52.874701 8183 raft_consensus.cc:348] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.874850 8183 raft_consensus.cc:374] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:52.874902 8183 raft_consensus.cc:729] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953edcf192924724ab15ed83aa6156bf, State: Initialized, Role: FOLLOWER
I20260619 12:09:52.874888 7715 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 12:09:52.875037 8183 consensus_queue.cc:260] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.875289 8183 ts_tablet_manager.cc:1428] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 12:09:52.875321 8158 heartbeater.cc:503] Master 127.0.201.62:35305 was elected leader, sending a full tablet report...
I20260619 12:09:52.875686 7744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a4785b9c63e4fdba05b1257ea44c8af. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.875775 7744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:09:52.877044 8185 tablet_bootstrap.cc:654] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:52.877328 8185 log.cc:824] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:52.878052 8184 tablet_bootstrap.cc:492] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: No bootstrap required, opened a new log
I20260619 12:09:52.878052 8185 tablet_bootstrap.cc:492] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: No bootstrap required, opened a new log
I20260619 12:09:52.878122 8184 ts_tablet_manager.cc:1397] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 12:09:52.878122 8185 ts_tablet_manager.cc:1397] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 12:09:52.879989 8184 raft_consensus.cc:348] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.879989 8185 raft_consensus.cc:348] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.880143 8184 raft_consensus.cc:374] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:52.880143 8185 raft_consensus.cc:374] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:52.880178 8185 raft_consensus.cc:729] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3ed0457e54445c8ddf7e804eee797c, State: Initialized, Role: FOLLOWER
I20260619 12:09:52.880178 8184 raft_consensus.cc:729] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c149b0fb6b94ede882ed006d1442da6, State: Initialized, Role: FOLLOWER
I20260619 12:09:52.880293 8185 consensus_queue.cc:260] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.880293 8184 consensus_queue.cc:260] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.880534 8184 ts_tablet_manager.cc:1428] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:09:52.880566 7885 heartbeater.cc:503] Master 127.0.201.62:35305 was elected leader, sending a full tablet report...
I20260619 12:09:52.880735 8022 heartbeater.cc:503] Master 127.0.201.62:35305 was elected leader, sending a full tablet report...
I20260619 12:09:52.880534 8185 ts_tablet_manager.cc:1428] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260619 12:09:52.934414 8024 tablet.cc:2369] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:52.961758 8195 raft_consensus.cc:482] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:52.961864 8195 raft_consensus.cc:504] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.962275 8195 leader_election.cc:288] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477), 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203)
I20260619 12:09:52.965260 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.962481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:56619 (local address 127.0.201.3:42477)
0619 12:09:52.962538 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.962540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.962549 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:52.962576 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.962578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.962589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.962656 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.962824 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.963479 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.964501 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.964967 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.965019 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.965027 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.965183 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.965208 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:09:52.965268 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.962403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:42477 (local address 127.0.201.2:56619)
0619 12:09:52.962457 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.962469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.962510 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.962667 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.962668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.962670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.962747 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.962749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.963550 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.963551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.964407 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.964410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.965031 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.965032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.965105 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.965108 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.965164 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.965199 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:09:52.965592 8111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2c06a833ec4780bfdc07bed06045b8" candidate_uuid: "2c149b0fb6b94ede882ed006d1442da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953edcf192924724ab15ed83aa6156bf" is_pre_election: true
I20260619 12:09:52.965758 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.962531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:32947 (local address 127.0.201.1:33203)
0619 12:09:52.962580 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.962582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.962614 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:09:52.962654 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.962655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.962667 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.962715 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:52.963233 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.963832 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.964828 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.965404 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.965461 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.965471 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.965662 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.965699 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:09:52.965763 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.962478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33203 (local address 127.0.201.2:32947)
0619 12:09:52.962528 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:52.962538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:52.962576 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:52.962740 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:52.962741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:52.962743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:52.963118 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.963120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.963921 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.963921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.964737 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:52.964739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.965514 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.965515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:52.965589 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.965591 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:52.965659 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:09:52.965692 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 12:09:52.965759 8111 raft_consensus.cc:2454] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c149b0fb6b94ede882ed006d1442da6 in term 0.
I20260619 12:09:52.965976 7911 leader_election.cc:302] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c149b0fb6b94ede882ed006d1442da6, 953edcf192924724ab15ed83aa6156bf; no voters:
I20260619 12:09:52.966137 8195 raft_consensus.cc:2781] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:52.966202 8195 raft_consensus.cc:482] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:52.966149 7838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2c06a833ec4780bfdc07bed06045b8" candidate_uuid: "2c149b0fb6b94ede882ed006d1442da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3ed0457e54445c8ddf7e804eee797c" is_pre_election: true
I20260619 12:09:52.966238 8195 raft_consensus.cc:3037] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:52.966297 7838 raft_consensus.cc:2454] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c149b0fb6b94ede882ed006d1442da6 in term 0.
I20260619 12:09:52.967001 8195 raft_consensus.cc:504] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.967137 8195 leader_election.cc:288] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [CANDIDATE]: Term 1 election: Requested vote from peers 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477), 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203)
I20260619 12:09:52.967321 7838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2c06a833ec4780bfdc07bed06045b8" candidate_uuid: "2c149b0fb6b94ede882ed006d1442da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3ed0457e54445c8ddf7e804eee797c"
I20260619 12:09:52.967402 7838 raft_consensus.cc:3037] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:52.967610 8111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2c06a833ec4780bfdc07bed06045b8" candidate_uuid: "2c149b0fb6b94ede882ed006d1442da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953edcf192924724ab15ed83aa6156bf"
I20260619 12:09:52.967681 8111 raft_consensus.cc:3037] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:52.968567 8111 raft_consensus.cc:2454] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c149b0fb6b94ede882ed006d1442da6 in term 1.
I20260619 12:09:52.968676 7838 raft_consensus.cc:2454] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c149b0fb6b94ede882ed006d1442da6 in term 1.
I20260619 12:09:52.968755 7911 leader_election.cc:302] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c149b0fb6b94ede882ed006d1442da6, 953edcf192924724ab15ed83aa6156bf; no voters:
I20260619 12:09:52.968863 8195 raft_consensus.cc:2781] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:52.968997 8195 raft_consensus.cc:686] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 LEADER]: Becoming Leader. State: Replica: 2c149b0fb6b94ede882ed006d1442da6, State: Running, Role: LEADER
I20260619 12:09:52.969202 8195 consensus_queue.cc:237] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } }
I20260619 12:09:52.970016 6919 catalog_manager.cc:5495] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "2c149b0fb6b94ede882ed006d1442da6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } health_report { overall_health: HEALTHY } } }
I20260619 12:09:52.970616 7480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: NON_VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: true } } }
I20260619 12:09:52.970731 7480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.971436 7669 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: attempting to promote NON_VOTER 3ff91b1f4a204e76887ea38e5512aa96 to VOTER
I20260619 12:09:52.971793 7669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } }
I20260619 12:09:52.972215 7528 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Refusing update from remote peer 1a4785b9c63e4fdba05b1257ea44c8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 12:09:52.972215 7715 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 LEARNER]: Refusing update from remote peer 1a4785b9c63e4fdba05b1257ea44c8af: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 12:09:52.972383 7479 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 12:09:52.972476 7480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [LEADER]: Connected to new peer: Peer: permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 12:09:52.973064 7479 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.973223 7528 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.973253 7715 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.973941 7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.974037 7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.974104 7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.974159 7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.974112 7744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.974197 7744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:52.974206 7479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.974269 7479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:52.974504 8199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:52.974601 8199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:52.980814 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.978880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.0.201.2:44847)
0619 12:09:52.978932 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.978934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.978942 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.978975 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.978975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.978990 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.979050 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:52.979125 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.979604 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.980276 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.980444 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.980490 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:52.980603 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:52.980692 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:52.980742 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:52.980758 (+ 16us) server_negotiation.cc:299] Negotiation successful
0619 12:09:52.980771 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:09:52.982286 7838 raft_consensus.cc:1261] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Refusing update from remote peer 2c149b0fb6b94ede882ed006d1442da6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:52.982295 8111 raft_consensus.cc:1261] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Refusing update from remote peer 2c149b0fb6b94ede882ed006d1442da6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:52.982606 8196 consensus_queue.cc:1035] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:52.982703 8195 consensus_queue.cc:1035] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:52.985442 8202 mvcc.cc:205] Tried to move back new op lower bound from 7298543587253211136 to 7298543587202609152. Current Snapshot: MvccSnapshot[applied={T|T < 7298543587253211136}]
I20260619 12:09:52.988991 6919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:52.989091 6919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:52.990362 6919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a4785b9c63e4fdba05b1257ea44c8af: Sending DeleteTablet for 3 replicas of tablet db2c06a833ec4780bfdc07bed06045b8
I20260619 12:09:52.990655 8091 tablet_service.cc:1514] Processing DeleteTablet for tablet db2c06a833ec4780bfdc07bed06045b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:52 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:33036
I20260619 12:09:52.990680 7818 tablet_service.cc:1514] Processing DeleteTablet for tablet db2c06a833ec4780bfdc07bed06045b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:52 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:59210
I20260619 12:09:52.990900 7955 tablet_service.cc:1514] Processing DeleteTablet for tablet db2c06a833ec4780bfdc07bed06045b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:52 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:60254
I20260619 12:09:52.991309 8213 tablet_replica.cc:330] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: stopping tablet replica
I20260619 12:09:52.991400 8213 raft_consensus.cc:2229] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:52.991506 8213 raft_consensus.cc:2258] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:52.991719 8211 tablet_replica.cc:330] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: stopping tablet replica
I20260619 12:09:52.991806 8211 raft_consensus.cc:2229] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:52.991910 8211 raft_consensus.cc:2258] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:52.991930 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 6888
I20260619 12:09:52.992683 8213 ts_tablet_manager.cc:1905] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:52.994240 8213 ts_tablet_manager.cc:1918] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:52.994314 8213 log.cc:1196] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/db2c06a833ec4780bfdc07bed06045b8
I20260619 12:09:52.994583 8213 ts_tablet_manager.cc:1939] T db2c06a833ec4780bfdc07bed06045b8 P 2c149b0fb6b94ede882ed006d1442da6: Deleting consensus metadata
I20260619 12:09:52.996001 8212 tablet_replica.cc:330] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: stopping tablet replica
I20260619 12:09:52.996153 8212 raft_consensus.cc:2229] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:52.996295 8212 raft_consensus.cc:2258] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:52.996748 8212 ts_tablet_manager.cc:1905] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:52.996790 8211 ts_tablet_manager.cc:1905] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:09:52.997337 7910 connection.cc:535] server connection from 127.0.0.1:60254 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:09:52.998327 8211 ts_tablet_manager.cc:1918] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:52.998425 8211 log.cc:1196] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/db2c06a833ec4780bfdc07bed06045b8
I20260619 12:09:52.998759 8211 ts_tablet_manager.cc:1939] T db2c06a833ec4780bfdc07bed06045b8 P 953edcf192924724ab15ed83aa6156bf: Deleting consensus metadata
W20260619 12:09:52.999114 8044 connection.cc:441] server connection from 127.0.0.1:33036 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33036 (request call id 1) could send its response
I20260619 12:09:52.999372 8212 ts_tablet_manager.cc:1918] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:52.999516 8212 log.cc:1196] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/db2c06a833ec4780bfdc07bed06045b8
I20260619 12:09:52.999881 8212 ts_tablet_manager.cc:1939] T db2c06a833ec4780bfdc07bed06045b8 P 5b3ed0457e54445c8ddf7e804eee797c: Deleting consensus metadata
W20260619 12:09:53.000259 7772 connection.cc:441] server connection from 127.0.0.1:59210 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59210 (request call id 1) could send its response
I20260619 12:09:53.004118 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.991332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45294 (local address 127.0.201.60:46105)
0619 12:09:52.991369 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.991371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.991379 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:52.991398 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.991399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.991409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.991457 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:52.995962 (+ 4505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.996687 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.997839 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:52.998129 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:52.998173 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:53.000453 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:53.000454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:53.000454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:53.000462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:53.002141 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:53.002604 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:53.002605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:53.002606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:53.002662 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:53.003161 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:53.003162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:53.003163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:53.003336 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:53.003474 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:53.003667 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 12:09:53.003743 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:09:53.006559 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.998893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.0.201.61:37357)
0619 12:09:52.998932 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.998934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:52.999465 (+ 531us) server_negotiation.cc:407] Connection header received
0619 12:09:52.999565 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:52.999566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:52.999577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:52.999606 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:53.000335 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:53.001125 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:53.002211 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:53.002508 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:53.002551 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:53.003909 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:53.003910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:53.003911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:53.003918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:53.005477 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:53.005806 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:53.005807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:53.005807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:53.005867 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:53.006068 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:53.006069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:53.006069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:53.006178 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:53.006278 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:53.006381 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:09:53.006443 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:09:53.007653 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:52.999231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45304 (local address 127.0.201.60:46105)
0619 12:09:52.999271 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:52.999273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:53.000709 (+ 1436us) server_negotiation.cc:407] Connection header received
0619 12:09:53.000807 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:53.000808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:53.000817 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:53.000844 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:53.001418 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:53.002133 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:53.003162 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:53.003500 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:53.003546 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:53.005035 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:53.005036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:53.005036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:53.005042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:53.006628 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:53.006995 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:53.006996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:53.006997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:53.007046 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:53.007252 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:53.007253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:53.007253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:53.007353 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:53.007420 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:53.007548 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 12:09:53.007601 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:53.986581 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:53.986317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:42983)
0619 12:09:53.986446 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:53.986522 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260619 12:09:53.986946 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:53.986586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:38147)
0619 12:09:53.986831 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:53.986896 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":46,"threads_started":1}
W20260619 12:09:53.986954 7885 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:35305 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
W20260619 12:09:53.987110 8158 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:35305 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:53.987893 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:53.987681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:60219)
0619 12:09:53.987801 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:53.987862 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
W20260619 12:09:53.988054 8022 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:35305 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:54.490942 8225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1a4785b9c63e4fdba05b1257ea44c8af)
I20260619 12:09:54.491065 8225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } }
I20260619 12:09:54.491389 8225 leader_election.cc:288] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305), 404e21e3716344df971cf67bcbb55655 (127.0.201.61:37357)
I20260619 12:09:54.491685 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.491477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59220)
0619 12:09:54.491588 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.491642 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":47,"threads_started":1}
W20260619 12:09:54.491822 7691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:54.492097 8230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1a4785b9c63e4fdba05b1257ea44c8af)
I20260619 12:09:54.492220 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.492123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59236)
0619 12:09:54.492175 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.492203 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:09:54.492223 8230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } }
W20260619 12:09:54.492286 7691 leader_election.cc:334] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305): Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:54.492503 8230 leader_election.cc:288] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305), 3ff91b1f4a204e76887ea38e5512aa96 (127.0.201.60:46105)
I20260619 12:09:54.492789 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.492573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59246)
0619 12:09:54.492691 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.492744 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
W20260619 12:09:54.492908 7504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:54.493324 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.493206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59260)
0619 12:09:54.493271 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.493297 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 12:09:54.493405 7504 leader_election.cc:334] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305): Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:54.495253 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.491599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45532 (local address 127.0.201.61:37357)
0619 12:09:54.491701 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.491704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.491737 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:09:54.492205 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.492206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.492219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.492279 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.492468 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.493252 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.494539 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.494937 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.495007 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.495018 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.495131 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.495169 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
I20260619 12:09:54.495273 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.491547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:37357 (local address 127.0.0.1:45532)
0619 12:09:54.491671 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.491683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.492013 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.492331 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.492333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.492336 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.492389 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.492392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.493324 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.493325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.494456 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.494459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.495019 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.495019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.495064 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.495068 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.495109 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.495160 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":18,"thread_start_us":62,"threads_started":1}
I20260619 12:09:54.495563 7528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ff91b1f4a204e76887ea38e5512aa96" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "404e21e3716344df971cf67bcbb55655" is_pre_election: true
I20260619 12:09:54.495664 7528 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff91b1f4a204e76887ea38e5512aa96 in term 1.
I20260619 12:09:54.495805 7689 leader_election.cc:302] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ff91b1f4a204e76887ea38e5512aa96, 404e21e3716344df971cf67bcbb55655; no voters: 1a4785b9c63e4fdba05b1257ea44c8af
I20260619 12:09:54.495853 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.492613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:46105 (local address 127.0.0.1:45318)
0619 12:09:54.492754 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.492763 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.492811 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.492982 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.492983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.492985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.493042 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.493044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.493808 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.493809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.494851 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.494853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.495606 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.495607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.495669 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.495674 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.495734 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.495781 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":47,"threads_started":1}
I20260619 12:09:54.495884 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.492713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.0.201.60:46105)
0619 12:09:54.492836 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.492838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.492846 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:54.492866 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.492867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.492876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.492934 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.493118 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.493720 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.494926 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.495547 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.495607 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.495623 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.495739 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.495777 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260619 12:09:54.495959 8225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 12:09:54.495988 8225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1a4785b9c63e4fdba05b1257ea44c8af)
I20260619 12:09:54.496008 8225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:09:54.496280 7715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "404e21e3716344df971cf67bcbb55655" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3ff91b1f4a204e76887ea38e5512aa96" is_pre_election: true
I20260619 12:09:54.496732 8225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } }
I20260619 12:09:54.496871 8225 leader_election.cc:288] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [CANDIDATE]: Term 2 election: Requested vote from peers 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305), 404e21e3716344df971cf67bcbb55655 (127.0.201.61:37357)
I20260619 12:09:54.496855 7715 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 404e21e3716344df971cf67bcbb55655 in current term 2: Already voted for candidate 3ff91b1f4a204e76887ea38e5512aa96 in this term.
I20260619 12:09:54.497011 7528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ff91b1f4a204e76887ea38e5512aa96" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "404e21e3716344df971cf67bcbb55655"
I20260619 12:09:54.497076 7528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:09:54.497097 7502 leader_election.cc:302] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 404e21e3716344df971cf67bcbb55655; no voters: 1a4785b9c63e4fdba05b1257ea44c8af, 3ff91b1f4a204e76887ea38e5512aa96
I20260619 12:09:54.497294 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.496948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59268)
0619 12:09:54.497244 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.497273 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281}
I20260619 12:09:54.497615 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.497548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59272)
0619 12:09:54.497575 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.497597 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 12:09:54.497694 7691 leader_election.cc:334] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305): Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
I20260619 12:09:54.497895 7528 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff91b1f4a204e76887ea38e5512aa96 in term 2.
I20260619 12:09:54.497972 8230 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:09:54.498075 7689 leader_election.cc:302] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ff91b1f4a204e76887ea38e5512aa96, 404e21e3716344df971cf67bcbb55655; no voters: 1a4785b9c63e4fdba05b1257ea44c8af
I20260619 12:09:54.498150 8225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:09:54.498271 8225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [term 2 LEADER]: Becoming Leader. State: Replica: 3ff91b1f4a204e76887ea38e5512aa96, State: Running, Role: LEADER
I20260619 12:09:54.498364 8225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } }
I20260619 12:09:54.498673 8235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff91b1f4a204e76887ea38e5512aa96. Latest consensus state: current_term: 2 leader_uuid: "3ff91b1f4a204e76887ea38e5512aa96" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:54.498772 8235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:54.498963 8237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:54.499439 8237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:54.499529 8237 catalog_manager.cc:1259] Loaded cluster ID: ff452963a7924aa88c4c69f334abe857
I20260619 12:09:54.499562 8237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:09:54.499716 8237 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:54.499806 8237 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Loaded TSK: 0
I20260619 12:09:54.499943 8237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:09:54.539237 7704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:54.540858 7704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:54.541885 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.541785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59284)
0619 12:09:54.541819 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.541856 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 12:09:54.541910 7528 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [term 2 FOLLOWER]: Refusing update from remote peer 3ff91b1f4a204e76887ea38e5512aa96: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 12:09:54.542176 8235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 12:09:54.542276 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.542160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.0.1:59286)
0619 12:09:54.542226 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.542255 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 12:09:54.542369 7691 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 -> Peer 1a4785b9c63e4fdba05b1257ea44c8af (127.0.201.62:35305): Couldn't send request to peer 1a4785b9c63e4fdba05b1257ea44c8af. Status: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:09:54.542888 8230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff91b1f4a204e76887ea38e5512aa96. Latest consensus state: current_term: 2 leader_uuid: "3ff91b1f4a204e76887ea38e5512aa96" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:54.542970 8230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:54.543488 8235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3ff91b1f4a204e76887ea38e5512aa96" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:54.543571 8235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:54.543610 8230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3ff91b1f4a204e76887ea38e5512aa96" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1a4785b9c63e4fdba05b1257ea44c8af" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35305 } } peers { permanent_uuid: "404e21e3716344df971cf67bcbb55655" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 37357 } attrs { promote: false } } peers { permanent_uuid: "3ff91b1f4a204e76887ea38e5512aa96" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 46105 } attrs { promote: false } } }
I20260619 12:09:54.543678 8230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404e21e3716344df971cf67bcbb55655 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:09:54.543843 8239 mvcc.cc:205] Tried to move back new op lower bound from 7298543593641861120 to 7298543593465909248. Current Snapshot: MvccSnapshot[applied={T|T < 7298543593641861120}]
I20260619 12:09:54.548870 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.545641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44847 (local address 127.0.0.1:60270)
0619 12:09:54.545877 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.545886 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.545942 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.546078 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.546079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.546081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.546128 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.546129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.546881 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.546882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.548015 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.548018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.548639 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548639 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.548698 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.548701 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.548753 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.548797 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226}
I20260619 12:09:54.548877 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.545693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.0.201.2:44847)
0619 12:09:54.545847 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.545850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.545935 (+ 85us) server_negotiation.cc:407] Connection header received
0619 12:09:54.545995 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.545996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.546007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.546076 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.546176 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.546781 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548104 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.548559 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548642 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.548655 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.548769 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.548810 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":56,"threads_started":1}
I20260619 12:09:54.549093 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.545569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:42477 (local address 127.0.0.1:33040)
0619 12:09:54.545622 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.545633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.545684 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.546413 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.546413 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.546415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.546443 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.546444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.547299 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.547300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.548073 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.548075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.548863 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.548931 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.548935 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.548966 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.549006 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":2}
I20260619 12:09:54.549186 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.545771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33203 (local address 127.0.0.1:59212)
0619 12:09:54.545910 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.545920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.545980 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.546173 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.546174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.546176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.546222 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.546223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.547129 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.547130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.548206 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.548208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.548972 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.549021 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.549024 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.549077 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.549114 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":29,"thread_start_us":77,"threads_started":1}
I20260619 12:09:54.549219 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.545814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.0.201.1:33203)
0619 12:09:54.545931 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.545933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.545966 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:09:54.546042 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.546042 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.546053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.546109 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.546302 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.547052 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548283 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.548869 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548930 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.548942 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.549103 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.549156 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260619 12:09:54.549345 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.545671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.0.201.3:42477)
0619 12:09:54.546291 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.546293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.546301 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:54.546324 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.546325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.546334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.546395 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.546500 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.547239 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.548172 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.548763 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.549213 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.549230 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.549250 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.549287 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":54,"threads_started":1}
I20260619 12:09:54.549461 7955 tablet_service.cc:1467] Processing CreateTablet for tablet f06e23b795204dc7a931f11582683710 (DEFAULT_TABLE table=test-table [id=bc79e478fe1d405bbf720a8010cb2809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:54.549598 7955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e23b795204dc7a931f11582683710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:54.549717 8091 tablet_service.cc:1467] Processing CreateTablet for tablet f06e23b795204dc7a931f11582683710 (DEFAULT_TABLE table=test-table [id=bc79e478fe1d405bbf720a8010cb2809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:54.549860 8091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e23b795204dc7a931f11582683710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:54.550930 8247 tablet_bootstrap.cc:492] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Bootstrap starting.
I20260619 12:09:54.550997 7818 tablet_service.cc:1467] Processing CreateTablet for tablet f06e23b795204dc7a931f11582683710 (DEFAULT_TABLE table=test-table [id=bc79e478fe1d405bbf720a8010cb2809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:54.551100 7818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f06e23b795204dc7a931f11582683710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:54.551396 8247 tablet_bootstrap.cc:654] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:54.551465 8248 tablet_bootstrap.cc:492] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Bootstrap starting.
I20260619 12:09:54.551857 8248 tablet_bootstrap.cc:654] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:54.552006 8247 tablet_bootstrap.cc:492] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: No bootstrap required, opened a new log
I20260619 12:09:54.552068 8247 ts_tablet_manager.cc:1397] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:54.552222 8247 raft_consensus.cc:348] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.552323 8247 raft_consensus.cc:374] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:54.552345 8247 raft_consensus.cc:729] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c149b0fb6b94ede882ed006d1442da6, State: Initialized, Role: FOLLOWER
I20260619 12:09:54.552390 8247 consensus_queue.cc:260] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.552553 8247 ts_tablet_manager.cc:1428] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:54.553030 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.552779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:48767)
0619 12:09:54.552938 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.552982 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260619 12:09:54.553215 8248 tablet_bootstrap.cc:492] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: No bootstrap required, opened a new log
I20260619 12:09:54.553232 8021 heartbeater.cc:503] Master 127.0.201.60:46105 was elected leader, sending a full tablet report...
I20260619 12:09:54.553268 8248 ts_tablet_manager.cc:1397] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:54.553391 8248 raft_consensus.cc:348] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.553486 8248 raft_consensus.cc:374] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:54.553519 8248 raft_consensus.cc:729] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953edcf192924724ab15ed83aa6156bf, State: Initialized, Role: FOLLOWER
I20260619 12:09:54.553654 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.553568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:40887)
0619 12:09:54.553609 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.553631 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 12:09:54.553609 8248 consensus_queue.cc:260] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.553884 8252 tablet_bootstrap.cc:492] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: Bootstrap starting.
I20260619 12:09:54.553802 8248 ts_tablet_manager.cc:1428] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:54.554301 8252 tablet_bootstrap.cc:654] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:54.554400 8255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.553966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:48351)
0619 12:09:54.554087 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.554141 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260619 12:09:54.554469 8157 heartbeater.cc:503] Master 127.0.201.60:46105 was elected leader, sending a full tablet report...
I20260619 12:09:54.554833 8252 tablet_bootstrap.cc:492] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: No bootstrap required, opened a new log
I20260619 12:09:54.554883 8252 ts_tablet_manager.cc:1397] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:54.554986 8252 raft_consensus.cc:348] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.555058 8252 raft_consensus.cc:374] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:54.555058 8255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.554855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:51489)
0619 12:09:54.555005 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.555033 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260619 12:09:54.555089 8252 raft_consensus.cc:729] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3ed0457e54445c8ddf7e804eee797c, State: Initialized, Role: FOLLOWER
I20260619 12:09:54.555130 8252 consensus_queue.cc:260] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.555315 8252 ts_tablet_manager.cc:1428] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:54.555722 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.555512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:40055)
0619 12:09:54.555644 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.555681 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":96,"threads_started":1}
I20260619 12:09:54.555819 7884 heartbeater.cc:503] Master 127.0.201.60:46105 was elected leader, sending a full tablet report...
I20260619 12:09:54.556051 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.555952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:48623)
0619 12:09:54.556003 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.556030 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 12:09:54.585490 8160 tablet.cc:2369] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:54.586463 8257 raft_consensus.cc:482] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:54.586531 8257 raft_consensus.cc:504] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.586865 8257 leader_election.cc:288] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477), 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847)
I20260619 12:09:54.590771 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.587139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47449 (local address 127.0.201.2:44847)
0619 12:09:54.587181 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.587183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.587428 (+ 245us) server_negotiation.cc:407] Connection header received
0619 12:09:54.587452 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.587453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.587463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.587501 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.587623 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.588314 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.589818 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.590377 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.590448 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.590459 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.590658 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.590704 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:09:54.590771 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.587165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:35051 (local address 127.0.201.3:42477)
0619 12:09:54.587233 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.587235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.587244 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:54.587287 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.587290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.587305 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.587353 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:54.587549 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.588319 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.589818 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.590377 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.590448 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.590459 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.590658 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.590703 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:09:54.590766 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.587052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:42477 (local address 127.0.201.1:35051)
0619 12:09:54.587110 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.587124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.587188 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.587377 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.587379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.587381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.587459 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.587461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.588394 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.588395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.589718 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.589721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.590524 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.590525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.590583 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.590585 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.590639 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.590678 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":39}
I20260619 12:09:54.590792 8259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.587052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44847 (local address 127.0.201.1:47449)
0619 12:09:54.587314 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.587329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:54.587403 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:54.587516 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:54.587517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:54.587518 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:54.587560 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.587561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.588397 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.588397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.589718 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:54.589721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.590497 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.590498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:54.590576 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.590581 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:09:54.590639 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:09:54.590710 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":137,"thread_start_us":129,"threads_started":1}
I20260619 12:09:54.591025 8111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06e23b795204dc7a931f11582683710" candidate_uuid: "5b3ed0457e54445c8ddf7e804eee797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953edcf192924724ab15ed83aa6156bf" is_pre_election: true
I20260619 12:09:54.591115 8111 raft_consensus.cc:2454] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3ed0457e54445c8ddf7e804eee797c in term 0.
I20260619 12:09:54.591347 7774 leader_election.cc:302] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3ed0457e54445c8ddf7e804eee797c, 953edcf192924724ab15ed83aa6156bf; no voters:
I20260619 12:09:54.591351 7975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06e23b795204dc7a931f11582683710" candidate_uuid: "5b3ed0457e54445c8ddf7e804eee797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c149b0fb6b94ede882ed006d1442da6" is_pre_election: true
I20260619 12:09:54.591472 7975 raft_consensus.cc:2454] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b3ed0457e54445c8ddf7e804eee797c in term 0.
I20260619 12:09:54.591517 8257 raft_consensus.cc:2781] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:54.591576 8257 raft_consensus.cc:482] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:54.591610 8257 raft_consensus.cc:3037] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:54.592295 8257 raft_consensus.cc:504] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.592440 8257 leader_election.cc:288] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [CANDIDATE]: Term 1 election: Requested vote from peers 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477), 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847)
I20260619 12:09:54.592566 8111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06e23b795204dc7a931f11582683710" candidate_uuid: "5b3ed0457e54445c8ddf7e804eee797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953edcf192924724ab15ed83aa6156bf"
I20260619 12:09:54.592590 7975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f06e23b795204dc7a931f11582683710" candidate_uuid: "5b3ed0457e54445c8ddf7e804eee797c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c149b0fb6b94ede882ed006d1442da6"
I20260619 12:09:54.592648 7975 raft_consensus.cc:3037] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:54.592648 8111 raft_consensus.cc:3037] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:54.593371 8111 raft_consensus.cc:2454] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3ed0457e54445c8ddf7e804eee797c in term 1.
I20260619 12:09:54.593415 7975 raft_consensus.cc:2454] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b3ed0457e54445c8ddf7e804eee797c in term 1.
I20260619 12:09:54.593519 7774 leader_election.cc:302] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b3ed0457e54445c8ddf7e804eee797c, 953edcf192924724ab15ed83aa6156bf; no voters:
I20260619 12:09:54.593600 8257 raft_consensus.cc:2781] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:54.593827 8257 raft_consensus.cc:686] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 LEADER]: Becoming Leader. State: Replica: 5b3ed0457e54445c8ddf7e804eee797c, State: Running, Role: LEADER
I20260619 12:09:54.593917 8257 consensus_queue.cc:237] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } }
I20260619 12:09:54.594488 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.594373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:60677)
0619 12:09:54.594430 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.594460 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 12:09:54.594903 7704 catalog_manager.cc:5495] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1). New cstate: current_term: 1 leader_uuid: "5b3ed0457e54445c8ddf7e804eee797c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b3ed0457e54445c8ddf7e804eee797c" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33203 } health_report { overall_health: HEALTHY } } }
I20260619 12:09:54.606180 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.604173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.0.201.1:33203)
0619 12:09:54.604226 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:54.604227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:54.604234 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:09:54.604262 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:54.604263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:54.604278 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:54.604342 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:54.604441 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.604917 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.605588 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:54.605762 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:54.605809 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:54.605936 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:54.606026 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:54.606077 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:54.606114 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 12:09:54.606129 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:09:54.607110 8111 raft_consensus.cc:1261] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Refusing update from remote peer 5b3ed0457e54445c8ddf7e804eee797c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:54.607249 7975 raft_consensus.cc:1261] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Refusing update from remote peer 5b3ed0457e54445c8ddf7e804eee797c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:54.607363 8257 consensus_queue.cc:1035] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [LEADER]: Connected to new peer: Peer: permanent_uuid: "953edcf192924724ab15ed83aa6156bf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:54.607556 8260 consensus_queue.cc:1035] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c149b0fb6b94ede882ed006d1442da6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:54.607997 8255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.607907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:33483)
0619 12:09:54.607939 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.607969 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260619 12:09:54.608675 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.608579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:48181)
0619 12:09:54.608618 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.608643 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 12:09:54.609544 8255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.609460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.3:45587)
0619 12:09:54.609494 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.609517 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 12:09:54.609692 8266 mvcc.cc:205] Tried to move back new op lower bound from 7298543593908932608 to 7298543593857318912. Current Snapshot: MvccSnapshot[applied={T|T < 7298543593908932608}]
I20260619 12:09:54.609737 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.609278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.1:50365)
0619 12:09:54.609688 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.609716 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398}
I20260619 12:09:54.610037 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.609955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:35305 (local address 127.0.201.2:60737)
0619 12:09:54.609990 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.610014 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 12:09:54.611820 7704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:54.611917 7704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:54.613183 7704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ff91b1f4a204e76887ea38e5512aa96: Sending DeleteTablet for 3 replicas of tablet f06e23b795204dc7a931f11582683710
I20260619 12:09:54.613481 8091 tablet_service.cc:1514] Processing DeleteTablet for tablet f06e23b795204dc7a931f11582683710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:54 UTC) from {username='oryx', principal='oryx/127.0.201.60@KRBTEST.COM'} at 127.0.0.1:33040
I20260619 12:09:54.613543 7955 tablet_service.cc:1514] Processing DeleteTablet for tablet f06e23b795204dc7a931f11582683710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:54 UTC) from {username='oryx', principal='oryx/127.0.201.60@KRBTEST.COM'} at 127.0.0.1:60270
I20260619 12:09:54.613569 7818 tablet_service.cc:1514] Processing DeleteTablet for tablet f06e23b795204dc7a931f11582683710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:54 UTC) from {username='oryx', principal='oryx/127.0.201.60@KRBTEST.COM'} at 127.0.0.1:59212
I20260619 12:09:54.613771 8276 tablet_replica.cc:330] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: stopping tablet replica
I20260619 12:09:54.613844 8276 raft_consensus.cc:2229] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:54.613926 8276 raft_consensus.cc:2258] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:54.613970 8274 tablet_replica.cc:330] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: stopping tablet replica
I20260619 12:09:54.614027 8274 raft_consensus.cc:2229] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:54.614065 8274 raft_consensus.cc:2258] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:54.614055 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 7666
I20260619 12:09:54.614214 8276 ts_tablet_manager.cc:1905] T f06e23b795204dc7a931f11582683710 P 5b3ed0457e54445c8ddf7e804eee797c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:54.614671 8275 tablet_replica.cc:330] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: stopping tablet replica
I20260619 12:09:54.614710 8275 raft_consensus.cc:2229] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:54.614737 8275 raft_consensus.cc:2258] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:54.615023 8275 ts_tablet_manager.cc:1905] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:54.615928 8274 ts_tablet_manager.cc:1905] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:54.616451 8275 ts_tablet_manager.cc:1918] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:54.616488 8275 log.cc:1196] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/f06e23b795204dc7a931f11582683710
I20260619 12:09:54.616674 8275 ts_tablet_manager.cc:1939] T f06e23b795204dc7a931f11582683710 P 2c149b0fb6b94ede882ed006d1442da6: Deleting consensus metadata
I20260619 12:09:54.617057 7691 catalog_manager.cc:4841] TS 2c149b0fb6b94ede882ed006d1442da6 (127.0.201.2:44847): tablet f06e23b795204dc7a931f11582683710 (table test-table [id=bc79e478fe1d405bbf720a8010cb2809]) successfully deleted
I20260619 12:09:54.617764 8274 ts_tablet_manager.cc:1918] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:54.617830 8274 log.cc:1196] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/f06e23b795204dc7a931f11582683710
I20260619 12:09:54.618048 8274 ts_tablet_manager.cc:1939] T f06e23b795204dc7a931f11582683710 P 953edcf192924724ab15ed83aa6156bf: Deleting consensus metadata
I20260619 12:09:54.618348 7692 catalog_manager.cc:4841] TS 953edcf192924724ab15ed83aa6156bf (127.0.201.3:42477): tablet f06e23b795204dc7a931f11582683710 (table test-table [id=bc79e478fe1d405bbf720a8010cb2809]) successfully deleted
I20260619 12:09:54.620827 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:54.620703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33203 (local address 127.0.0.1:59216)
0619 12:09:54.620759 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:54.620786 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:33203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:09:54.620903 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 7892
W20260619 12:09:54.620959 7691 catalog_manager.cc:4584] TS 5b3ed0457e54445c8ddf7e804eee797c (127.0.201.1:33203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f06e23b795204dc7a931f11582683710: Network error: Client connection negotiation failed: client connection to 127.0.201.1:33203: connect: Connection refused (error 111)
I20260619 12:09:54.627279 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 8028
I20260619 12:09:54.631982 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 7384
I20260619 12:09:54.637759 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 7572
2026-06-19T12:09:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4543 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:54 dist-test-slave-c910 krb5kdc[8279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:54 dist-test-slave-c910 krb5kdc[8279](info): set up 2 sockets
Jun 19 12:09:54 dist-test-slave-c910 krb5kdc[8279](info): commencing operation
krb5kdc: starting...
W20260619 12:09:55.679661 804 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:55 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab.
Entry for principal oryx/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab.
I20260619 12:09:55.801265 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44909
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab
--principal=oryx/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:44909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:55.886961 8292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:55.887157 8292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:55.887179 8292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:55.888350 8292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:55.888388 8292 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:09:55.888401 8292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:55.888414 8292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:55.888427 8292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:55.888438 8292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:55.889760 8292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:44909
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.62.keytab
--principal=oryx/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:55.889938 8292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:55.890100 8292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:55.894629 8297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:55.894726 8300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:55.894723 8298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:55.894852 8292 server_base.cc:1034] running on GCE node
I20260619 12:09:55.895079 8292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:55.895404 8292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:55.896572 8292 hybrid_clock.cc:648] HybridClock initialized: now 1781870995896567 us; error 35 us; skew 500 ppm
Jun 19 12:09:55 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870995, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:55.898978 8292 init.cc:377] Logged in from keytab as oryx/127.0.201.62@KRBTEST.COM (short username oryx)
I20260619 12:09:55.899441 8292 webserver.cc:460] Webserver started at http://127.0.201.62:35077/ using document root <none> and password file <none>
I20260619 12:09:55.899653 8292 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:55.899718 8292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:55.899873 8292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:55.900843 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "c0e3edfad7e44fc48cb895d326c652e9"
format_stamp: "Formatted at 2026-06-19 12:09:55 on dist-test-slave-c910"
server_key: "735a39aad915c5593fa59fc500e2b8cc"
server_key_iv: "c336501b38f6847a755dfa2b08f3a982"
server_key_version: "encryptionkey@0"
I20260619 12:09:55.901186 8292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "c0e3edfad7e44fc48cb895d326c652e9"
format_stamp: "Formatted at 2026-06-19 12:09:55 on dist-test-slave-c910"
server_key: "735a39aad915c5593fa59fc500e2b8cc"
server_key_iv: "c336501b38f6847a755dfa2b08f3a982"
server_key_version: "encryptionkey@0"
I20260619 12:09:55.902460 8292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:55.903165 8307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:55.903412 8292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:55.903519 8292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "c0e3edfad7e44fc48cb895d326c652e9"
format_stamp: "Formatted at 2026-06-19 12:09:55 on dist-test-slave-c910"
server_key: "735a39aad915c5593fa59fc500e2b8cc"
server_key_iv: "c336501b38f6847a755dfa2b08f3a982"
server_key_version: "encryptionkey@0"
I20260619 12:09:55.903599 8292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:55.920965 8292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:55.922868 8292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:55.923025 8292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:55.927601 8292 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:44909
I20260619 12:09:55.927649 8369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:44909 every 8 connection(s)
I20260619 12:09:55.927959 8292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:55.928521 8370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:55.930648 8370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Bootstrap starting.
I20260619 12:09:55.931277 8370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:55.931566 8370 log.cc:824] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:55.932237 8370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: No bootstrap required, opened a new log
I20260619 12:09:55.934221 8370 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e3edfad7e44fc48cb895d326c652e9" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44909 } }
I20260619 12:09:55.934346 8370 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:55.934420 8370 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e3edfad7e44fc48cb895d326c652e9, State: Initialized, Role: FOLLOWER
I20260619 12:09:55.934576 8370 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e3edfad7e44fc48cb895d326c652e9" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44909 } }
I20260619 12:09:55.934655 8370 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:55.934705 8370 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:55.934752 8370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:55.935302 8370 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e3edfad7e44fc48cb895d326c652e9" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44909 } }
I20260619 12:09:55.935403 8370 leader_election.cc:302] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0e3edfad7e44fc48cb895d326c652e9; no voters:
I20260619 12:09:55.935551 8370 leader_election.cc:288] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:55.935580 8373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:55.935654 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 8292
I20260619 12:09:55.935702 8373 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [term 1 LEADER]: Becoming Leader. State: Replica: c0e3edfad7e44fc48cb895d326c652e9, State: Running, Role: LEADER
I20260619 12:09:55.935762 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:55.935776 8373 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e3edfad7e44fc48cb895d326c652e9" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44909 } }
I20260619 12:09:55.935808 8370 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:55.935967 804 external_mini_cluster.cc:1428] Setting key 59701380f33fef73158fb5ef2ac892e6
I20260619 12:09:55.936733 8375 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0e3edfad7e44fc48cb895d326c652e9. Latest consensus state: current_term: 1 leader_uuid: "c0e3edfad7e44fc48cb895d326c652e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e3edfad7e44fc48cb895d326c652e9" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44909 } } }
I20260619 12:09:55.936836 8375 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:55.937233 8384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:55.937234 8374 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0e3edfad7e44fc48cb895d326c652e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e3edfad7e44fc48cb895d326c652e9" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44909 } } }
I20260619 12:09:55.937362 8374 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:55.937789 8384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:55.939396 8384 catalog_manager.cc:1347] Generated new cluster ID: f1395453ab334c2d81cdc54a0fda8897
I20260619 12:09:55.939447 8384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:55 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:55.947661 8384 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:09:55.947970 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:55.936433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.0.201.62:44909)
0619 12:09:55.937306 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:55.937327 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:55.937365 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:09:55.937493 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:55.937500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:55.937553 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:55.937728 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:55.938503 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:55.939859 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:55.940898 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:55.941510 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:55.941622 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:55.944285 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:55.944289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:55.944289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:55.944300 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:55.946190 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:55.946598 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:55.946600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:55.946601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:55.946703 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:55.946919 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:55.946920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:55.946920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:55.947181 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:55.947507 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:55.947630 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 12:09:55.947771 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":66,"threads_started":1}
I20260619 12:09:55.948707 8384 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:09:55.952450 8384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Generated new TSK 0
I20260619 12:09:55.952610 8384 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab.
Entry for principal oryx/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab.
I20260619 12:09:55.982712 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:44909
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:56.078711 8396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:56.078928 8396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:56.078969 8396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:56.080205 8396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:56.080294 8396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:56.080331 8396 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:56.080369 8396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:09:56.081825 8396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:44909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:56.082082 8396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:56.082306 8396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:56.088109 8401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:56.088186 8402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:56.088316 8404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:56.088316 8396 server_base.cc:1034] running on GCE node
I20260619 12:09:56.088529 8396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:56.088771 8396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:56.089936 8396 hybrid_clock.cc:648] HybridClock initialized: now 1781870996089916 us; error 34 us; skew 500 ppm
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:56.092483 8396 init.cc:377] Logged in from keytab as oryx/127.0.201.1@KRBTEST.COM (short username oryx)
I20260619 12:09:56.092912 8396 webserver.cc:460] Webserver started at http://127.0.201.1:38003/ using document root <none> and password file <none>
I20260619 12:09:56.093154 8396 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:56.093199 8396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:56.093369 8396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:56.094393 8396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "f3b044033b9edad6a74dfe3ad959300e"
server_key_iv: "1b971770d465b7a406f433513bbeb031"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.094739 8396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "f3b044033b9edad6a74dfe3ad959300e"
server_key_iv: "1b971770d465b7a406f433513bbeb031"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.096077 8396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:09:56.096784 8411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.096989 8396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:09:56.097087 8396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "f3b044033b9edad6a74dfe3ad959300e"
server_key_iv: "1b971770d465b7a406f433513bbeb031"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.097163 8396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:56.113701 8396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:56.115571 8396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:56.115732 8396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:56.116231 8396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:56.116271 8396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.116294 8396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:56.116308 8396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.122669 8396 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:36515
I20260619 12:09:56.122742 8526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:36515 every 8 connection(s)
I20260619 12:09:56.123010 8396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:56.126626 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.116731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:34885 (local address 127.0.201.62:44909)
0619 12:09:56.116806 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.116808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.117005 (+ 197us) server_negotiation.cc:407] Connection header received
0619 12:09:56.117501 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.117502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.117513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.117583 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.118437 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.119330 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.120875 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.121203 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.121288 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.123826 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.123828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.123828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.123835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.125270 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.125682 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.125683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.125684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.125743 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.125987 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.125988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.125988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.126104 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.126179 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.126442 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.126535 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:56.126773 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.116706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.201.1:34885)
0619 12:09:56.116869 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.116907 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.117389 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.117665 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.117668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.118032 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:56.118324 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.118331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.119420 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.119421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.120802 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.120805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.121296 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.121297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.121433 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.122011 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:56.122021 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:56.123721 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:56.125376 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.125382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.125384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.125595 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.125809 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.125809 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.125810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.125921 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.126169 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:56.126175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:56.126314 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.126419 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.126561 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260619 12:09:56.127266 8527 heartbeater.cc:347] Connected to a master server at 127.0.201.62:44909
I20260619 12:09:56.127347 8527 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:56.127522 8527 heartbeater.cc:511] Master 127.0.201.62:44909 requested a full tablet report, sending...
I20260619 12:09:56.128006 8324 ts_manager.cc:194] Registered new tserver with Master: 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515)
I20260619 12:09:56.128311 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 8396
I20260619 12:09:56.128413 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:09:56.128600 804 external_mini_cluster.cc:1428] Setting key d99a6e2911b4f0fc8d67d410f3731a24
I20260619 12:09:56.128778 8324 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:34885
I20260619 12:09:56.134940 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.128876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.201.1:38545)
0619 12:09:56.129092 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.129100 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.129171 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.129274 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.129275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.129461 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:56.129512 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.129514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.130195 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.130196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.131063 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.131065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.131499 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.131500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.131570 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.131964 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:56.131969 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:56.132341 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:56.134028 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.134029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.134029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.134202 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.134420 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.134420 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.134421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.134467 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.134728 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:56.134729 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:56.134760 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.134811 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.134878 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":69,"threads_started":1}
I20260619 12:09:56.135052 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.128922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:38545 (local address 127.0.201.62:44909)
0619 12:09:56.128959 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.128961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.129160 (+ 199us) server_negotiation.cc:407] Connection header received
0619 12:09:56.129220 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.129220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.129228 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.129270 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.129581 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.130108 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.131170 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.131433 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.131497 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.132449 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.132450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.132451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.132458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.133948 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.134278 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.134279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.134280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.134339 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.134530 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.134531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.134531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.134663 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.134741 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.134831 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.134914 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab.
Entry for principal oryx/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab.
I20260619 12:09:56.154131 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:44909
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:56.235432 8536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:56.235639 8536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:56.235661 8536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:56.236920 8536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:56.236959 8536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:56.236979 8536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:56.237030 8536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:09:56.238483 8536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:44909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:56.238718 8536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:56.238932 8536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:56.243739 8541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:56.243784 8544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:56.243784 8542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:56.244092 8536 server_base.cc:1034] running on GCE node
I20260619 12:09:56.244294 8536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:56.244479 8536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:56.245612 8536 hybrid_clock.cc:648] HybridClock initialized: now 1781870996245597 us; error 32 us; skew 500 ppm
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:56.248049 8536 init.cc:377] Logged in from keytab as oryx/127.0.201.2@KRBTEST.COM (short username oryx)
I20260619 12:09:56.248446 8536 webserver.cc:460] Webserver started at http://127.0.201.2:41767/ using document root <none> and password file <none>
I20260619 12:09:56.248677 8536 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:56.248744 8536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:56.248903 8536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:56.249869 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "6a790a214c144f23aca7512cc8f20256"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "52e7c61c56e2944a81a61508892afb0d"
server_key_iv: "1112e7d408eb43afefd26fad87aee757"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.250204 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "6a790a214c144f23aca7512cc8f20256"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "52e7c61c56e2944a81a61508892afb0d"
server_key_iv: "1112e7d408eb43afefd26fad87aee757"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.251495 8536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:09:56.252247 8551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.252415 8536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:56.252507 8536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "6a790a214c144f23aca7512cc8f20256"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "52e7c61c56e2944a81a61508892afb0d"
server_key_iv: "1112e7d408eb43afefd26fad87aee757"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.252585 8536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:56.266238 8536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:56.268150 8536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:56.268311 8536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:56.268755 8536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:56.268821 8536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.268864 8536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:56.268891 8536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:56.276050 8536 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:43639
I20260619 12:09:56.276166 8666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:43639 every 8 connection(s)
I20260619 12:09:56.276420 8536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:09:56.278364 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 8536
I20260619 12:09:56.278479 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:09:56.278685 804 external_mini_cluster.cc:1428] Setting key 78cdec367cc8be60ab8c3f22a300d127
I20260619 12:09:56.279053 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.269236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:44715 (local address 127.0.201.62:44909)
0619 12:09:56.269319 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.269323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.269524 (+ 201us) server_negotiation.cc:407] Connection header received
0619 12:09:56.269948 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.269949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.269962 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.270046 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.270867 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.271455 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.272442 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.272705 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.272790 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.276074 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.276075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.276076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.276083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.277565 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.277997 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.277999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.277999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.278063 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.278315 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.278315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.278316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.278439 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.278517 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.278852 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.278946 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:09:56.279239 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.269189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.201.2:44715)
0619 12:09:56.269397 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.269423 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.269857 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.270124 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.270128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.270496 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:56.270783 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.270789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.271556 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.271557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.272350 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.272352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.272798 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.272799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.272925 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.273412 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:56.273423 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:56.275966 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:56.277679 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.277685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.277687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.277905 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.278129 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.278130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.278130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.278246 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.278514 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:56.278519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:56.278680 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.278817 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.278995 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":70,"threads_started":1}
I20260619 12:09:56.279870 8667 heartbeater.cc:347] Connected to a master server at 127.0.201.62:44909
I20260619 12:09:56.279959 8667 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:56.280134 8667 heartbeater.cc:511] Master 127.0.201.62:44909 requested a full tablet report, sending...
I20260619 12:09:56.280493 8323 ts_manager.cc:194] Registered new tserver with Master: 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639)
I20260619 12:09:56.280998 8323 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:44715
I20260619 12:09:56.287277 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.281108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:50891 (local address 127.0.201.62:44909)
0619 12:09:56.281141 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.281143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.281270 (+ 127us) server_negotiation.cc:407] Connection header received
0619 12:09:56.281320 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.281321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.281331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.281386 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.281767 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.282212 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.283339 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.283517 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.283571 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.284578 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.284579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.284579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.284585 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.285887 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.286487 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.286487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.286487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.286533 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.286753 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.286754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.286754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.286873 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.286922 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.287113 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.287163 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 12:09:56.287384 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.281054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.201.2:50891)
0619 12:09:56.281197 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.281209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.281268 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.281388 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.281389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.281607 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:56.281686 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.281688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.282311 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.282312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.283254 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.283257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.283595 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.283596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.283685 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.284061 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:56.284066 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:56.284484 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:56.286156 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.286157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.286157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.286404 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.286609 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.286610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.286610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.286686 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.286944 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:56.286945 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:56.286999 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.287101 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.287244 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for oryx/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab.
Entry for principal oryx/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab.
I20260619 12:09:56.305388 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:44909
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:56.387636 8676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:56.387863 8676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:56.387899 8676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:56.389087 8676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:56.389151 8676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:56.389184 8676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:09:56.389246 8676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:09:56.390686 8676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.201.62:44909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:56.390944 8676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:56.391162 8676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:56.396066 8681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:56.396056 8682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:56.396054 8684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:56.396370 8676 server_base.cc:1034] running on GCE node
I20260619 12:09:56.396574 8676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:56.396795 8676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:56.397931 8676 hybrid_clock.cc:648] HybridClock initialized: now 1781870996397913 us; error 47 us; skew 500 ppm
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:56.400594 8676 init.cc:377] Logged in from keytab as oryx/127.0.201.3@KRBTEST.COM (short username oryx)
I20260619 12:09:56.401033 8676 webserver.cc:460] Webserver started at http://127.0.201.3:37989/ using document root <none> and password file <none>
I20260619 12:09:56.401270 8676 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:56.401320 8676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:56.401404 8676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:56.402340 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "c5c11c3ca4174c878281f8851d5a12f2"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "3ca9482883fadc89723a88845105fc70"
server_key_iv: "e0bc28ce3218fdae4b164a1772c9041b"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.402621 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "c5c11c3ca4174c878281f8851d5a12f2"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "3ca9482883fadc89723a88845105fc70"
server_key_iv: "e0bc28ce3218fdae4b164a1772c9041b"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.403877 8676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:56.404536 8691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.404748 8676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:56.404814 8676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "c5c11c3ca4174c878281f8851d5a12f2"
format_stamp: "Formatted at 2026-06-19 12:09:56 on dist-test-slave-c910"
server_key: "3ca9482883fadc89723a88845105fc70"
server_key_iv: "e0bc28ce3218fdae4b164a1772c9041b"
server_key_version: "encryptionkey@0"
I20260619 12:09:56.404860 8676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:56.416363 8676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:56.418531 8676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:56.418637 8676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:56.419124 8676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:09:56.419176 8676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.419281 8676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:09:56.419315 8676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:56.426410 8676 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:45333
I20260619 12:09:56.426560 8806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:45333 every 8 connection(s)
I20260619 12:09:56.426847 8676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:09:56.429183 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.419509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:39083 (local address 127.0.201.62:44909)
0619 12:09:56.419552 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.419554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.419775 (+ 221us) server_negotiation.cc:407] Connection header received
0619 12:09:56.420377 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.420379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.420392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.420438 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.421110 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.421606 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.422652 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.422846 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.422930 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.426006 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.426007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.426008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.426081 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.427830 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.428294 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.428295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.428296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.428355 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.428611 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.428612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.428612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.428752 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.428815 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.429029 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.429095 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:09:56.429350 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.419508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.201.3:39083)
0619 12:09:56.419649 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.419691 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.420268 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.420521 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.420524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.420808 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:56.421016 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.421021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.421727 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.421728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.422538 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.422540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.422940 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.422941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.423061 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.423607 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:56.423615 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:56.425778 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:56.427915 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.427921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.427924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.428192 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.428425 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.428426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.428426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.428544 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.428819 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:56.428823 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:56.428935 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.429013 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.429126 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260619 12:09:56.429801 8807 heartbeater.cc:347] Connected to a master server at 127.0.201.62:44909
I20260619 12:09:56.429883 8807 heartbeater.cc:464] Registering TS with master...
I20260619 12:09:56.430055 8807 heartbeater.cc:511] Master 127.0.201.62:44909 requested a full tablet report, sending...
I20260619 12:09:56.430382 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 8676
I20260619 12:09:56.430460 8323 ts_manager.cc:194] Registered new tserver with Master: c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:56.430493 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:09:56.430698 804 external_mini_cluster.cc:1428] Setting key 16836202a9d0f6a35810a2ae7b2fd65a
I20260619 12:09:56.430949 8323 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:39083
I20260619 12:09:56.431656 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:09:56.437377 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.431221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.201.3:48635)
0619 12:09:56.431339 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.431350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.431407 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.431531 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.431532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.431714 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:56.431768 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.431770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.432625 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.432626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.433138 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.433139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.433601 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.433602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.433673 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.434054 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:56.434058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:56.434454 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:56.436232 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.436233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.436233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.436384 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.436623 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:56.436624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:56.436625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:56.436686 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:56.437020 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:56.437020 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:56.437071 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.437142 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.437260 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":66,"threads_started":1}
I20260619 12:09:56.437377 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.431309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:48635 (local address 127.0.201.62:44909)
0619 12:09:56.431351 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.431353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.431393 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:09:56.431462 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.431463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.431472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.431524 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.431847 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.432527 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.433217 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.433507 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.433575 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.434570 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.434572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.434572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.434579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.436145 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.436458 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.436459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.436459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.436519 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.436763 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.436764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.436765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.436927 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.436989 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.437176 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.437262 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:09:56.453632 8377 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:56.448473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.0.201.62:44909)
0619 12:09:56.448534 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.448536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.448544 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:56.448563 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.448564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.448573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.448628 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.449219 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.449749 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.450502 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.450707 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.450788 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.453543 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34328: BlockingRecv error: recv got EOF from 127.0.0.1:34328 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 12:09:56 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870996, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:56.460803 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.454424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.0.201.62:44909)
0619 12:09:56.454474 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.454475 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.454591 (+ 116us) server_negotiation.cc:407] Connection header received
0619 12:09:56.454636 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.454637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.454687 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.454727 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:56.455228 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.455687 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.456364 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.456552 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.456603 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.458594 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:56.458596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:56.458596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:56.458604 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:56.459901 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.460181 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.460182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.460182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.460233 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:56.460382 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:56.460382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:56.460382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:56.460470 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:56.460541 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.460630 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.460687 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:09:56.462337 8324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:56.464051 8324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:56.470072 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.466190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:43639 (local address 127.0.0.1:33598)
0619 12:09:56.466341 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.466380 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.466443 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.466679 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.466682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.466684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:56.466743 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.466745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.467840 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.467841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.468834 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.468836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.469765 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.469766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.469859 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.469874 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.469950 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.470002 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
I20260619 12:09:56.470112 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.466290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.0.201.2:43639)
0619 12:09:56.466510 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.466515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.466527 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:09:56.466571 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.466573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.466589 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.466676 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:56.466845 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.467732 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.468942 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.469684 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.469752 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.469827 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.469974 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.470013 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
I20260619 12:09:56.470533 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.466222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45333 (local address 127.0.0.1:53702)
0619 12:09:56.466765 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.466775 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.466828 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.467042 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.467043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.467045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:56.467096 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.467097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.468413 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.468414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.469367 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.469369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.470310 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.470310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.470354 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.470357 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.470435 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.470471 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260619 12:09:56.470592 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.466528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.0.201.3:45333)
0619 12:09:56.466755 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.466759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.466844 (+ 85us) server_negotiation.cc:407] Connection header received
0619 12:09:56.466889 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.466891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.466907 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.466981 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:56.467181 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.468323 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.469455 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.470135 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.470204 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.470270 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.470456 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.470501 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260619 12:09:56.470815 8601 tablet_service.cc:1467] Processing CreateTablet for tablet e5ff550580034eef8f13466df1e5e3e5 (DEFAULT_TABLE table=test-table [id=4b1753a83bc64b5f9c05f67ab5698d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:56.471172 8601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ff550580034eef8f13466df1e5e3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:56.471228 8741 tablet_service.cc:1467] Processing CreateTablet for tablet e5ff550580034eef8f13466df1e5e3e5 (DEFAULT_TABLE table=test-table [id=4b1753a83bc64b5f9c05f67ab5698d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:56.471468 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.466384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36515 (local address 127.0.0.1:51848)
0619 12:09:56.466795 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.466803 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.466880 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.467151 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.467152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.467154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:56.467186 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.467187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.468459 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.468460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.469478 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.469480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.471237 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.471238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.471301 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.471305 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.471355 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.471397 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":80,"threads_started":1}
I20260619 12:09:56.471552 8741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ff550580034eef8f13466df1e5e3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:56.471750 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.466479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.0.201.1:36515)
0619 12:09:56.466650 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.466655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.466960 (+ 305us) server_negotiation.cc:407] Connection header received
0619 12:09:56.466999 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.467001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.467015 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.467085 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:56.467291 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.468369 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.469577 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.470937 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.471185 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.471357 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.471382 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.471476 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260619 12:09:56.472277 8461 tablet_service.cc:1467] Processing CreateTablet for tablet e5ff550580034eef8f13466df1e5e3e5 (DEFAULT_TABLE table=test-table [id=4b1753a83bc64b5f9c05f67ab5698d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:56.472579 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ff550580034eef8f13466df1e5e3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:56.474970 8835 tablet_bootstrap.cc:492] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Bootstrap starting.
I20260619 12:09:56.475307 8836 tablet_bootstrap.cc:492] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Bootstrap starting.
I20260619 12:09:56.475728 8835 tablet_bootstrap.cc:654] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:56.475901 8836 tablet_bootstrap.cc:654] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:56.476064 8835 log.cc:824] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:56.476226 8836 log.cc:824] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:56.476436 8837 tablet_bootstrap.cc:492] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Bootstrap starting.
I20260619 12:09:56.476826 8835 tablet_bootstrap.cc:492] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: No bootstrap required, opened a new log
I20260619 12:09:56.476881 8835 ts_tablet_manager.cc:1397] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:56.476959 8836 tablet_bootstrap.cc:492] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: No bootstrap required, opened a new log
I20260619 12:09:56.477039 8836 ts_tablet_manager.cc:1397] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:09:56.478459 8835 raft_consensus.cc:348] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.478583 8835 raft_consensus.cc:374] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:56.478605 8835 raft_consensus.cc:729] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c11c3ca4174c878281f8851d5a12f2, State: Initialized, Role: FOLLOWER
I20260619 12:09:56.478571 8836 raft_consensus.cc:348] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.478739 8835 consensus_queue.cc:260] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.478758 8836 raft_consensus.cc:374] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:56.478814 8836 raft_consensus.cc:729] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Initialized, Role: FOLLOWER
I20260619 12:09:56.478927 8836 consensus_queue.cc:260] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.479007 8807 heartbeater.cc:503] Master 127.0.201.62:44909 was elected leader, sending a full tablet report...
I20260619 12:09:56.479228 8836 ts_tablet_manager.cc:1428] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:09:56.479007 8835 ts_tablet_manager.cc:1428] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:56.479239 8667 heartbeater.cc:503] Master 127.0.201.62:44909 was elected leader, sending a full tablet report...
I20260619 12:09:56.479928 8837 tablet_bootstrap.cc:654] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:56.480288 8837 log.cc:824] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:56.481014 8837 tablet_bootstrap.cc:492] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: No bootstrap required, opened a new log
I20260619 12:09:56.481104 8837 ts_tablet_manager.cc:1397] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 12:09:56.482542 8837 raft_consensus.cc:348] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.482700 8837 raft_consensus.cc:374] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:56.482724 8837 raft_consensus.cc:729] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Initialized, Role: FOLLOWER
I20260619 12:09:56.482801 8837 consensus_queue.cc:260] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.482980 8837 ts_tablet_manager.cc:1428] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:09:56.483034 8527 heartbeater.cc:503] Master 127.0.201.62:44909 was elected leader, sending a full tablet report...
W20260619 12:09:56.526980 8668 tablet.cc:2369] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:56.601429 8841 raft_consensus.cc:482] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:56.601547 8841 raft_consensus.cc:504] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.601866 8841 leader_election.cc:288] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:56.604805 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.602148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:58053 (local address 127.0.201.3:45333)
0619 12:09:56.602203 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.602205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.602214 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:09:56.602241 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.602242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.602257 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.602323 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:56.602473 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.602954 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.604063 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.604430 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.604506 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.604516 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.604694 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.604729 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:09:56.604810 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.602087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:51273 (local address 127.0.201.1:36515)
0619 12:09:56.602236 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.602238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.602293 (+ 55us) server_negotiation.cc:407] Connection header received
0619 12:09:56.602368 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.602369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.602380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.602450 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:56.602603 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.603062 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.604096 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.604460 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.604540 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.604549 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.604706 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.604738 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260619 12:09:56.604841 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.602049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45333 (local address 127.0.201.2:58053)
0619 12:09:56.602112 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.602123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.602167 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.602329 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.602330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.602331 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:56.602403 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.602405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.603043 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.603044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.603976 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.603978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.604515 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.604516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.604604 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.604607 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.604687 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.604725 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260619 12:09:56.605042 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.602050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36515 (local address 127.0.201.2:51273)
0619 12:09:56.602227 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:56.602238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:56.602303 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:56.602453 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:56.602453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:56.602455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:56.602534 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.602536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.603141 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.603142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.603996 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:56.603998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.604541 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.604542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:56.604613 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.604616 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:09:56.604699 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:09:56.604739 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":14,"thread_start_us":101,"threads_started":1}
I20260619 12:09:56.605185 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ff550580034eef8f13466df1e5e3e5" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2" is_pre_election: true
I20260619 12:09:56.605343 8761 raft_consensus.cc:2454] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 0.
I20260619 12:09:56.605370 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ff550580034eef8f13466df1e5e3e5" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" is_pre_election: true
I20260619 12:09:56.605556 8481 raft_consensus.cc:2454] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 0.
I20260619 12:09:56.605578 8554 leader_election.cc:302] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a790a214c144f23aca7512cc8f20256, c5c11c3ca4174c878281f8851d5a12f2; no voters:
I20260619 12:09:56.605691 8841 raft_consensus.cc:2781] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:56.605746 8841 raft_consensus.cc:482] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:56.605778 8841 raft_consensus.cc:3037] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:56.606817 8841 raft_consensus.cc:504] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.606959 8841 leader_election.cc:288] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 election: Requested vote from peers 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:56.607097 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ff550580034eef8f13466df1e5e3e5" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18"
I20260619 12:09:56.607112 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ff550580034eef8f13466df1e5e3e5" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2"
I20260619 12:09:56.607178 8761 raft_consensus.cc:3037] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:56.607194 8481 raft_consensus.cc:3037] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:56.607928 8761 raft_consensus.cc:2454] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 1.
I20260619 12:09:56.607936 8481 raft_consensus.cc:2454] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 1.
I20260619 12:09:56.608085 8554 leader_election.cc:302] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a790a214c144f23aca7512cc8f20256, c5c11c3ca4174c878281f8851d5a12f2; no voters:
I20260619 12:09:56.608207 8841 raft_consensus.cc:2781] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:56.608389 8841 raft_consensus.cc:686] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 1 LEADER]: Becoming Leader. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Running, Role: LEADER
I20260619 12:09:56.608492 8841 consensus_queue.cc:237] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.609287 8324 catalog_manager.cc:5495] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a790a214c144f23aca7512cc8f20256 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "6a790a214c144f23aca7512cc8f20256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:56.618541 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.616197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.0.201.2:43639)
0619 12:09:56.616243 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:56.616245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:56.616253 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:09:56.616329 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:56.616330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:56.616348 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:56.616427 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:56.616572 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.617027 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.617803 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:56.617994 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:56.618066 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:56.618239 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:56.618335 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:56.618424 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:56.618443 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 12:09:56.618463 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:09:56.620360 8481 raft_consensus.cc:1261] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Refusing update from remote peer 6a790a214c144f23aca7512cc8f20256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:56.620471 8761 raft_consensus.cc:1261] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Refusing update from remote peer 6a790a214c144f23aca7512cc8f20256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:56.620663 8845 consensus_queue.cc:1035] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:56.620764 8841 consensus_queue.cc:1035] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:56.623389 8848 mvcc.cc:205] Tried to move back new op lower bound from 7298543602154168320 to 7298543602109059072. Current Snapshot: MvccSnapshot[applied={T|T < 7298543602154168320}]
I20260619 12:09:56.623631 8849 mvcc.cc:205] Tried to move back new op lower bound from 7298543602154168320 to 7298543602109059072. Current Snapshot: MvccSnapshot[applied={T|T < 7298543602154168320}]
I20260619 12:09:56.625546 8324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:56.625631 8324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:56.626387 8324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Sending DeleteTablet for 3 replicas of tablet e5ff550580034eef8f13466df1e5e3e5
I20260619 12:09:56.626696 8601 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ff550580034eef8f13466df1e5e3e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:56 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:33598
I20260619 12:09:56.626693 8741 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ff550580034eef8f13466df1e5e3e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:56 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:53702
I20260619 12:09:56.626718 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ff550580034eef8f13466df1e5e3e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:56 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:51848
I20260619 12:09:56.627065 8860 tablet_replica.cc:330] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: stopping tablet replica
I20260619 12:09:56.627161 8860 raft_consensus.cc:2229] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:56.627162 8324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:09:56.627293 8860 raft_consensus.cc:2258] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:09:56.627344 8324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:56.627836 8860 ts_tablet_manager.cc:1905] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:56.628548 8461 tablet_service.cc:1467] Processing CreateTablet for tablet eefac816e4234d0b877ac1f6bd47d9f0 (DEFAULT_TABLE table=test-table [id=621b673f4a4d4a389442c086d6efa382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:56.628573 8741 tablet_service.cc:1467] Processing CreateTablet for tablet eefac816e4234d0b877ac1f6bd47d9f0 (DEFAULT_TABLE table=test-table [id=621b673f4a4d4a389442c086d6efa382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:56.628607 8601 tablet_service.cc:1467] Processing CreateTablet for tablet eefac816e4234d0b877ac1f6bd47d9f0 (DEFAULT_TABLE table=test-table [id=621b673f4a4d4a389442c086d6efa382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:56.628659 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eefac816e4234d0b877ac1f6bd47d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:56.628679 8741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eefac816e4234d0b877ac1f6bd47d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:56.628684 8601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eefac816e4234d0b877ac1f6bd47d9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:56.629603 8837 tablet_bootstrap.cc:492] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Bootstrap starting.
I20260619 12:09:56.629974 8835 tablet_bootstrap.cc:492] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Bootstrap starting.
I20260619 12:09:56.629992 8836 tablet_bootstrap.cc:492] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Bootstrap starting.
I20260619 12:09:56.630014 8837 tablet_bootstrap.cc:654] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:56.630420 8836 tablet_bootstrap.cc:654] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:56.630616 8835 tablet_bootstrap.cc:654] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:56.631062 8836 tablet_bootstrap.cc:492] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: No bootstrap required, opened a new log
I20260619 12:09:56.631109 8836 ts_tablet_manager.cc:1397] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:56.631260 8836 raft_consensus.cc:348] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.631358 8836 raft_consensus.cc:374] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:56.631392 8836 raft_consensus.cc:729] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Initialized, Role: FOLLOWER
I20260619 12:09:56.631449 8836 consensus_queue.cc:260] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.631547 8836 ts_tablet_manager.cc:1428] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.631626 8835 tablet_bootstrap.cc:492] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: No bootstrap required, opened a new log
I20260619 12:09:56.631695 8835 ts_tablet_manager.cc:1397] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:56.631836 8835 raft_consensus.cc:348] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.631910 8835 raft_consensus.cc:374] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:56.631932 8835 raft_consensus.cc:729] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c11c3ca4174c878281f8851d5a12f2, State: Initialized, Role: FOLLOWER
I20260619 12:09:56.631945 8858 tablet_replica.cc:330] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: stopping tablet replica
I20260619 12:09:56.631976 8837 tablet_bootstrap.cc:492] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: No bootstrap required, opened a new log
I20260619 12:09:56.631970 8835 consensus_queue.cc:260] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.632021 8837 ts_tablet_manager.cc:1397] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:09:56.632030 8858 raft_consensus.cc:2229] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:56.632048 8835 ts_tablet_manager.cc:1428] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.632121 8858 raft_consensus.cc:2258] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:56.632149 8837 raft_consensus.cc:348] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.632215 8837 raft_consensus.cc:374] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:56.632236 8837 raft_consensus.cc:729] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Initialized, Role: FOLLOWER
I20260619 12:09:56.632279 8837 consensus_queue.cc:260] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.632364 8837 ts_tablet_manager.cc:1428] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:56.633122 8858 ts_tablet_manager.cc:1905] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:56.633986 8860 ts_tablet_manager.cc:1918] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:56.634060 8860 log.cc:1196] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/e5ff550580034eef8f13466df1e5e3e5
I20260619 12:09:56.634375 8860 ts_tablet_manager.cc:1939] T e5ff550580034eef8f13466df1e5e3e5 P 6a790a214c144f23aca7512cc8f20256: Deleting consensus metadata
I20260619 12:09:56.634773 8309 catalog_manager.cc:4841] TS 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639): tablet e5ff550580034eef8f13466df1e5e3e5 (table test-table [id=4b1753a83bc64b5f9c05f67ab5698d17]) successfully deleted
I20260619 12:09:56.634862 8858 ts_tablet_manager.cc:1918] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:56.634932 8858 log.cc:1196] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/e5ff550580034eef8f13466df1e5e3e5
I20260619 12:09:56.635201 8858 ts_tablet_manager.cc:1939] T e5ff550580034eef8f13466df1e5e3e5 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting consensus metadata
I20260619 12:09:56.635596 8309 catalog_manager.cc:4841] TS 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515): tablet e5ff550580034eef8f13466df1e5e3e5 (table test-table [id=4b1753a83bc64b5f9c05f67ab5698d17]) successfully deleted
I20260619 12:09:56.639304 8859 tablet_replica.cc:330] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: stopping tablet replica
I20260619 12:09:56.639415 8859 raft_consensus.cc:2229] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:56.639511 8859 raft_consensus.cc:2258] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:56.639879 8859 ts_tablet_manager.cc:1905] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:56.641147 8859 ts_tablet_manager.cc:1918] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:56.641224 8859 log.cc:1196] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/e5ff550580034eef8f13466df1e5e3e5
I20260619 12:09:56.641446 8859 ts_tablet_manager.cc:1939] T e5ff550580034eef8f13466df1e5e3e5 P c5c11c3ca4174c878281f8851d5a12f2: Deleting consensus metadata
I20260619 12:09:56.641813 8310 catalog_manager.cc:4841] TS c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333): tablet e5ff550580034eef8f13466df1e5e3e5 (table test-table [id=4b1753a83bc64b5f9c05f67ab5698d17]) successfully deleted
W20260619 12:09:56.677548 8808 tablet.cc:2369] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:09:56.874045 8528 tablet.cc:2369] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:09:56.969621 8855 raft_consensus.cc:482] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:56.969736 8855 raft_consensus.cc:504] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.969957 8855 leader_election.cc:288] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:56.970185 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefac816e4234d0b877ac1f6bd47d9f0" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" is_pre_election: true
I20260619 12:09:56.970213 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefac816e4234d0b877ac1f6bd47d9f0" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2" is_pre_election: true
I20260619 12:09:56.970311 8481 raft_consensus.cc:2454] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 0.
I20260619 12:09:56.970330 8761 raft_consensus.cc:2454] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 0.
I20260619 12:09:56.970484 8554 leader_election.cc:302] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a790a214c144f23aca7512cc8f20256, c5c11c3ca4174c878281f8851d5a12f2; no voters:
I20260619 12:09:56.970630 8855 raft_consensus.cc:2781] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:56.970680 8855 raft_consensus.cc:482] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:56.970700 8855 raft_consensus.cc:3037] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:56.971500 8855 raft_consensus.cc:504] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.971629 8855 leader_election.cc:288] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 election: Requested vote from peers 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:56.971788 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefac816e4234d0b877ac1f6bd47d9f0" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18"
I20260619 12:09:56.971839 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefac816e4234d0b877ac1f6bd47d9f0" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2"
I20260619 12:09:56.971889 8481 raft_consensus.cc:3037] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:56.971900 8761 raft_consensus.cc:3037] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:56.972555 8481 raft_consensus.cc:2454] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 1.
I20260619 12:09:56.972662 8761 raft_consensus.cc:2454] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 1.
I20260619 12:09:56.972760 8553 leader_election.cc:302] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1cdbc5ed44771ad2dba69d62d4c18, 6a790a214c144f23aca7512cc8f20256; no voters:
I20260619 12:09:56.972849 8855 raft_consensus.cc:2781] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:56.972911 8855 raft_consensus.cc:686] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 1 LEADER]: Becoming Leader. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Running, Role: LEADER
I20260619 12:09:56.972965 8855 consensus_queue.cc:237] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:56.973515 8324 catalog_manager.cc:5495] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a790a214c144f23aca7512cc8f20256 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "6a790a214c144f23aca7512cc8f20256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870995, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.62@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:57.006608 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.999829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44909 (local address 127.0.0.1:34346)
0619 12:09:57.000015 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.000029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:57.000099 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:57.000250 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:57.000251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:57.000551 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:09:57.000595 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.000597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.001272 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.001273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.001862 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.001864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.002182 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.002183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.002223 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.002714 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:09:57.002719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:09:57.004085 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:09:57.005597 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:57.005602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:57.005605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:57.005833 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:57.006005 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:09:57.006005 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:09:57.006006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:09:57.006063 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:09:57.006296 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:09:57.006301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:09:57.006341 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 12:09:57.006440 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:09:57.006534 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260619 12:09:57.006755 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:56.999928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34346 (local address 127.0.201.62:44909)
0619 12:09:57.000126 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.000127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.000149 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:09:57.000169 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.000170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.000179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.000231 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:57.000669 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.001200 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.001937 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.002118 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.002166 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.004166 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:57.004167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:57.004167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:57.004173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:57.005525 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:57.005897 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:57.005898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:57.005898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:57.005948 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:57.006125 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:57.006125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:57.006126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:57.006237 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:57.006287 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.006474 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.006603 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":175,"thread_start_us":134,"threads_started":1}
I20260619 12:09:57.008888 8324 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:34346:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:09:57.009110 8324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:57.010547 8461 tablet_service.cc:1467] Processing CreateTablet for tablet 0186d59c690742a1a5879db3c118ce91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=384dfce2a58a49cd93318430c61578e4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:57.010550 8601 tablet_service.cc:1467] Processing CreateTablet for tablet 0186d59c690742a1a5879db3c118ce91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=384dfce2a58a49cd93318430c61578e4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:57.010571 8741 tablet_service.cc:1467] Processing CreateTablet for tablet 0186d59c690742a1a5879db3c118ce91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=384dfce2a58a49cd93318430c61578e4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:09:57.010694 8741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0186d59c690742a1a5879db3c118ce91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.010695 8601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0186d59c690742a1a5879db3c118ce91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.010736 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0186d59c690742a1a5879db3c118ce91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.012019 8837 tablet_bootstrap.cc:492] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Bootstrap starting.
I20260619 12:09:57.012167 8836 tablet_bootstrap.cc:492] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: Bootstrap starting.
I20260619 12:09:57.012177 8835 tablet_bootstrap.cc:492] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: Bootstrap starting.
I20260619 12:09:57.012446 8837 tablet_bootstrap.cc:654] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.012516 8836 tablet_bootstrap.cc:654] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.012578 8835 tablet_bootstrap.cc:654] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.013080 8836 tablet_bootstrap.cc:492] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: No bootstrap required, opened a new log
I20260619 12:09:57.013136 8836 ts_tablet_manager.cc:1397] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:57.013140 8835 tablet_bootstrap.cc:492] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: No bootstrap required, opened a new log
I20260619 12:09:57.013151 8837 tablet_bootstrap.cc:492] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: No bootstrap required, opened a new log
I20260619 12:09:57.013190 8837 ts_tablet_manager.cc:1397] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:57.013195 8835 ts_tablet_manager.cc:1397] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:57.013248 8836 raft_consensus.cc:348] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.013324 8836 raft_consensus.cc:374] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.013348 8836 raft_consensus.cc:729] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.013334 8837 raft_consensus.cc:348] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.013348 8835 raft_consensus.cc:348] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.013398 8837 raft_consensus.cc:374] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.013420 8837 raft_consensus.cc:729] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.013391 8836 consensus_queue.cc:260] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.013422 8835 raft_consensus.cc:374] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.013447 8835 raft_consensus.cc:729] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c11c3ca4174c878281f8851d5a12f2, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.013460 8837 consensus_queue.cc:260] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.013487 8836 ts_tablet_manager.cc:1428] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.013504 8835 consensus_queue.cc:260] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.013545 8837 ts_tablet_manager.cc:1428] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.013597 8835 ts_tablet_manager.cc:1428] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:57.013592 8855 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.013656 8855 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.013625 8843 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.013716 8843 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.013706 8840 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.013773 8840 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.042579 8843 raft_consensus.cc:482] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:57.042662 8843 raft_consensus.cc:504] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.043061 8843 leader_election.cc:288] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:57.046041 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.043288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:50879 (local address 127.0.201.2:43639)
0619 12:09:57.043334 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.043336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.043482 (+ 146us) server_negotiation.cc:407] Connection header received
0619 12:09:57.043553 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.043554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.043563 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.043626 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:57.043797 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.044273 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.045330 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.045690 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.045769 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.045780 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.045930 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.045967 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:57.046041 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.043330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:55543 (local address 127.0.201.3:45333)
0619 12:09:57.043376 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.043379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.043585 (+ 206us) server_negotiation.cc:407] Connection header received
0619 12:09:57.043667 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.043668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.043677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.043717 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:09:57.043875 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.044349 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.045336 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.045690 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.045772 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.045781 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.045938 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.045972 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:09:57.046068 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.043253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:43639 (local address 127.0.201.1:50879)
0619 12:09:57.043397 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.043415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:57.043493 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:57.043642 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:57.043643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:57.043645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:57.043728 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.043729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.044353 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.044355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.045234 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.045236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.045768 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.045769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.045848 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.045852 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:57.045925 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:09:57.045964 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":45,"thread_start_us":74,"threads_started":1}
I20260619 12:09:57.046070 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.043288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45333 (local address 127.0.201.1:55543)
0619 12:09:57.043508 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.043517 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:57.043592 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:57.043731 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:57.043732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:57.043733 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:09:57.043806 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.043808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.044436 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.044438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.045240 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.045243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.045773 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.045774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.045854 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.045858 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:09:57.045921 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:09:57.045962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260619 12:09:57.046382 8621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0186d59c690742a1a5879db3c118ce91" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a790a214c144f23aca7512cc8f20256" is_pre_election: true
I20260619 12:09:57.046523 8621 raft_consensus.cc:2454] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 0.
I20260619 12:09:57.046538 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0186d59c690742a1a5879db3c118ce91" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2" is_pre_election: true
I20260619 12:09:57.046615 8761 raft_consensus.cc:2454] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 0.
I20260619 12:09:57.046715 8413 leader_election.cc:302] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1cdbc5ed44771ad2dba69d62d4c18, 6a790a214c144f23aca7512cc8f20256; no voters:
I20260619 12:09:57.046826 8843 raft_consensus.cc:2781] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:57.046867 8843 raft_consensus.cc:482] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:57.046892 8843 raft_consensus.cc:3037] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.047580 8843 raft_consensus.cc:504] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.047741 8843 leader_election.cc:288] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 election: Requested vote from peers 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:57.047850 8621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0186d59c690742a1a5879db3c118ce91" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a790a214c144f23aca7512cc8f20256"
I20260619 12:09:57.047921 8621 raft_consensus.cc:3037] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.047927 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0186d59c690742a1a5879db3c118ce91" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2"
I20260619 12:09:57.048019 8761 raft_consensus.cc:3037] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.048578 8621 raft_consensus.cc:2454] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 1.
I20260619 12:09:57.048683 8761 raft_consensus.cc:2454] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 1.
I20260619 12:09:57.048787 8413 leader_election.cc:302] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1cdbc5ed44771ad2dba69d62d4c18, 6a790a214c144f23aca7512cc8f20256; no voters:
I20260619 12:09:57.048877 8843 raft_consensus.cc:2781] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:57.049031 8843 raft_consensus.cc:686] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Running, Role: LEADER
I20260619 12:09:57.049125 8843 consensus_queue.cc:237] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.049487 8877 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: TxnStatusTablet state changed. Reason: New leader 2fd1cdbc5ed44771ad2dba69d62d4c18. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.049592 8877 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:57.049777 8879 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:09:57.049909 8879 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:09:57.049887 8322 catalog_manager.cc:5495] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1). New cstate: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:57.061185 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.058584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.0.201.1:36515)
0619 12:09:57.058641 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.058643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.058731 (+ 88us) server_negotiation.cc:407] Connection header received
0619 12:09:57.058825 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.058826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.058843 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.058906 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:57.059063 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.059606 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.060382 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.060590 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.060665 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.060852 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.060954 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.061031 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.061088 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.061108 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:09:57.061198 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.058513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36515 (local address 127.0.0.1:51858)
0619 12:09:57.058616 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.058629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:57.058756 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:57.058918 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:57.058919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:57.058920 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:57.058992 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.058994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.059707 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.059708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.060291 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.060293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.060665 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.060666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.060729 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.060750 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.061026 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.061034 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:09:57.061074 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:09:57.061129 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:09:57.062446 8761 raft_consensus.cc:1261] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Refusing update from remote peer 2fd1cdbc5ed44771ad2dba69d62d4c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.062578 8621 raft_consensus.cc:1261] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Refusing update from remote peer 2fd1cdbc5ed44771ad2dba69d62d4c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.062674 8877 consensus_queue.cc:1035] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.062808 8843 consensus_queue.cc:1035] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.063234 8840 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: TxnStatusTablet state changed. Reason: New leader 2fd1cdbc5ed44771ad2dba69d62d4c18. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.063333 8840 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.063369 8855 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: TxnStatusTablet state changed. Reason: New leader 2fd1cdbc5ed44771ad2dba69d62d4c18. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.063453 8855 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.064102 8843 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.064180 8843 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:57.064227 8843 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.064273 8843 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 2fd1cdbc5ed44771ad2dba69d62d4c18: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:09:57.064339 8855 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.064433 8855 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P 6a790a214c144f23aca7512cc8f20256: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.064848 8851 mvcc.cc:205] Tried to move back new op lower bound from 7298543603965706240 to 7298543603913748480. Current Snapshot: MvccSnapshot[applied={T|T < 7298543603965706240}]
I20260619 12:09:57.064915 8840 tablet_replica.cc:439] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } } }
I20260619 12:09:57.064991 8840 tablet_replica.cc:442] T 0186d59c690742a1a5879db3c118ce91 P c5c11c3ca4174c878281f8851d5a12f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:09:57.070283 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.067849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:53667 (local address 127.0.201.1:36515)
0619 12:09:57.067884 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.067886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.067953 (+ 67us) server_negotiation.cc:407] Connection header received
0619 12:09:57.068016 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.068016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.068024 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.068069 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:57.068181 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.068599 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.069692 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.069859 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.069912 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.070087 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.070109 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.070147 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.070223 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.070236 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 12:09:57.070353 8887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.067752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36515 (local address 127.0.201.2:53667)
0619 12:09:57.067896 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.067906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:57.067961 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:57.068081 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:57.068082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:57.068084 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:57.068129 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.068130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.068668 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.068668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.069604 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.069607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.069934 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.069935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.069992 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.070006 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.070167 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.070169 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:09:57.070222 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:09:57.070273 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260619 12:09:57.072127 8761 raft_consensus.cc:1261] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Refusing update from remote peer 6a790a214c144f23aca7512cc8f20256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.072139 8481 raft_consensus.cc:1261] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Refusing update from remote peer 6a790a214c144f23aca7512cc8f20256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.072337 8845 consensus_queue.cc:1035] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.072419 8855 consensus_queue.cc:1035] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.080089 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.077023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:48165 (local address 127.0.201.2:43639)
0619 12:09:57.077060 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.077062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.077117 (+ 55us) server_negotiation.cc:407] Connection header received
0619 12:09:57.077210 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.077211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.077222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.077250 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:57.077461 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.078163 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.079383 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.079592 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.079647 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.079878 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.079916 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.079957 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.080025 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.080041 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:09:57.080274 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.076896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:43639 (local address 127.0.201.1:48165)
0619 12:09:57.077060 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.077070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:09:57.077142 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:09:57.077319 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:09:57.077320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:09:57.077321 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:09:57.077391 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.077392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.078285 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.078286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.079283 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:09:57.079286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.079662 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.079663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:09:57.079728 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.079744 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.079971 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.079972 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 12:09:57.080023 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:09:57.080063 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260619 12:09:57.086314 8323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:57.086404 8323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:57.087074 8323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Sending DeleteTablet for 3 replicas of tablet eefac816e4234d0b877ac1f6bd47d9f0
I20260619 12:09:57.087358 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet eefac816e4234d0b877ac1f6bd47d9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:51848
I20260619 12:09:57.087370 8741 tablet_service.cc:1514] Processing DeleteTablet for tablet eefac816e4234d0b877ac1f6bd47d9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:53702
I20260619 12:09:57.087451 8601 tablet_service.cc:1514] Processing DeleteTablet for tablet eefac816e4234d0b877ac1f6bd47d9f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:33598
I20260619 12:09:57.087495 8859 tablet_replica.cc:330] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: stopping tablet replica
I20260619 12:09:57.087540 8858 tablet_replica.cc:330] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: stopping tablet replica
I20260619 12:09:57.087538 8860 tablet_replica.cc:330] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: stopping tablet replica
I20260619 12:09:57.087584 8858 raft_consensus.cc:2229] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:57.087587 8860 raft_consensus.cc:2229] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:57.087617 8858 raft_consensus.cc:2258] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.087559 8859 raft_consensus.cc:2229] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:57.087637 8860 raft_consensus.cc:2258] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.087657 8859 raft_consensus.cc:2258] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.087950 8860 ts_tablet_manager.cc:1905] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.088029 8859 ts_tablet_manager.cc:1905] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.088155 8858 ts_tablet_manager.cc:1905] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.089594 8860 ts_tablet_manager.cc:1918] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:57.089594 8858 ts_tablet_manager.cc:1918] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:57.089643 8858 log.cc:1196] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/eefac816e4234d0b877ac1f6bd47d9f0
I20260619 12:09:57.089643 8860 log.cc:1196] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/eefac816e4234d0b877ac1f6bd47d9f0
I20260619 12:09:57.089865 8860 ts_tablet_manager.cc:1939] T eefac816e4234d0b877ac1f6bd47d9f0 P 6a790a214c144f23aca7512cc8f20256: Deleting consensus metadata
I20260619 12:09:57.089865 8858 ts_tablet_manager.cc:1939] T eefac816e4234d0b877ac1f6bd47d9f0 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting consensus metadata
I20260619 12:09:57.089892 8859 ts_tablet_manager.cc:1918] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:57.089954 8859 log.cc:1196] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/eefac816e4234d0b877ac1f6bd47d9f0
I20260619 12:09:57.090204 8859 ts_tablet_manager.cc:1939] T eefac816e4234d0b877ac1f6bd47d9f0 P c5c11c3ca4174c878281f8851d5a12f2: Deleting consensus metadata
I20260619 12:09:57.090332 8309 catalog_manager.cc:4841] TS 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639): tablet eefac816e4234d0b877ac1f6bd47d9f0 (table test-table [id=621b673f4a4d4a389442c086d6efa382]) successfully deleted
I20260619 12:09:57.090500 8309 catalog_manager.cc:4841] TS 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515): tablet eefac816e4234d0b877ac1f6bd47d9f0 (table test-table [id=621b673f4a4d4a389442c086d6efa382]) successfully deleted
I20260619 12:09:57.090826 8310 catalog_manager.cc:4841] TS c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333): tablet eefac816e4234d0b877ac1f6bd47d9f0 (table test-table [id=621b673f4a4d4a389442c086d6efa382]) successfully deleted
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:09:57.111974 8871 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:09:57.105420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.0.201.62:44909)
0619 12:09:57.105488 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.105491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.105598 (+ 107us) server_negotiation.cc:407] Connection header received
0619 12:09:57.105697 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.105697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.105707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.105756 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:57.106502 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.107131 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.108305 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.108512 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.108580 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.111902 (+ 3322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34350: BlockingRecv error: recv got EOF from 127.0.0.1:34350 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM
I20260619 12:09:57.120103 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.112780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.0.201.62:44909)
0619 12:09:57.112818 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.112820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.112861 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:09:57.112931 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.112932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.112939 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.112979 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:57.113520 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.113994 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.114987 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.115163 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.115293 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.117509 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:09:57.117510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:09:57.117511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:09:57.117517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:09:57.118904 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:57.119357 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:57.119358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:57.119358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:57.119415 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:09:57.119612 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:09:57.119613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:09:57.119614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:09:57.119722 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:09:57.119829 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.119929 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.120003 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:09:57.121563 8323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:57.121743 8323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:57.123036 8461 tablet_service.cc:1467] Processing CreateTablet for tablet 51fce47dc7804b0cbc6624f733a2f34c (DEFAULT_TABLE table=test-table [id=5622e364cc7d402aa1996dd97400329c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:57.123093 8741 tablet_service.cc:1467] Processing CreateTablet for tablet 51fce47dc7804b0cbc6624f733a2f34c (DEFAULT_TABLE table=test-table [id=5622e364cc7d402aa1996dd97400329c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:57.123168 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51fce47dc7804b0cbc6624f733a2f34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.123257 8741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51fce47dc7804b0cbc6624f733a2f34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.124219 8835 tablet_bootstrap.cc:492] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Bootstrap starting.
I20260619 12:09:57.124500 8601 tablet_service.cc:1467] Processing CreateTablet for tablet 51fce47dc7804b0cbc6624f733a2f34c (DEFAULT_TABLE table=test-table [id=5622e364cc7d402aa1996dd97400329c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:57.124624 8601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51fce47dc7804b0cbc6624f733a2f34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.124651 8835 tablet_bootstrap.cc:654] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.125344 8835 tablet_bootstrap.cc:492] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: No bootstrap required, opened a new log
I20260619 12:09:57.125406 8835 ts_tablet_manager.cc:1397] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:57.125564 8835 raft_consensus.cc:348] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.125646 8835 raft_consensus.cc:374] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.125681 8835 raft_consensus.cc:729] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c11c3ca4174c878281f8851d5a12f2, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.125723 8835 consensus_queue.cc:260] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.125818 8835 ts_tablet_manager.cc:1428] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:57.125867 8837 tablet_bootstrap.cc:492] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Bootstrap starting.
I20260619 12:09:57.126127 8837 tablet_bootstrap.cc:654] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.126492 8836 tablet_bootstrap.cc:492] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Bootstrap starting.
I20260619 12:09:57.126778 8836 tablet_bootstrap.cc:654] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.127288 8837 tablet_bootstrap.cc:492] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: No bootstrap required, opened a new log
I20260619 12:09:57.127338 8837 ts_tablet_manager.cc:1397] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:57.127460 8837 raft_consensus.cc:348] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.127524 8837 raft_consensus.cc:374] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.127552 8837 raft_consensus.cc:729] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.127588 8837 consensus_queue.cc:260] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.127611 8836 tablet_bootstrap.cc:492] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: No bootstrap required, opened a new log
I20260619 12:09:57.127655 8836 ts_tablet_manager.cc:1397] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:57.127663 8837 ts_tablet_manager.cc:1428] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.127784 8836 raft_consensus.cc:348] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.127851 8836 raft_consensus.cc:374] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.127873 8836 raft_consensus.cc:729] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.127921 8836 consensus_queue.cc:260] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.128016 8836 ts_tablet_manager.cc:1428] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.193061 8877 raft_consensus.cc:482] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:57.193199 8877 raft_consensus.cc:504] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.193377 8877 leader_election.cc:288] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:57.193594 8621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51fce47dc7804b0cbc6624f733a2f34c" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a790a214c144f23aca7512cc8f20256" is_pre_election: true
I20260619 12:09:57.193711 8621 raft_consensus.cc:2454] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 0.
I20260619 12:09:57.193678 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51fce47dc7804b0cbc6624f733a2f34c" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2" is_pre_election: true
I20260619 12:09:57.193792 8761 raft_consensus.cc:2454] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 0.
I20260619 12:09:57.193892 8413 leader_election.cc:302] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1cdbc5ed44771ad2dba69d62d4c18, 6a790a214c144f23aca7512cc8f20256; no voters:
I20260619 12:09:57.194000 8877 raft_consensus.cc:2781] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:57.194070 8877 raft_consensus.cc:482] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:57.194111 8877 raft_consensus.cc:3037] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.194779 8877 raft_consensus.cc:504] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.194918 8877 leader_election.cc:288] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 election: Requested vote from peers 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:57.195066 8621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51fce47dc7804b0cbc6624f733a2f34c" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a790a214c144f23aca7512cc8f20256"
I20260619 12:09:57.195132 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51fce47dc7804b0cbc6624f733a2f34c" candidate_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2"
I20260619 12:09:57.195168 8621 raft_consensus.cc:3037] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.195231 8761 raft_consensus.cc:3037] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.195771 8621 raft_consensus.cc:2454] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 1.
I20260619 12:09:57.195948 8413 leader_election.cc:302] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1cdbc5ed44771ad2dba69d62d4c18, 6a790a214c144f23aca7512cc8f20256; no voters:
I20260619 12:09:57.195991 8761 raft_consensus.cc:2454] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd1cdbc5ed44771ad2dba69d62d4c18 in term 1.
I20260619 12:09:57.196056 8877 raft_consensus.cc:2781] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:57.196103 8877 raft_consensus.cc:686] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Running, Role: LEADER
I20260619 12:09:57.196161 8877 consensus_queue.cc:237] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.196767 8322 catalog_manager.cc:5495] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1). New cstate: current_term: 1 leader_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:57.219251 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.216953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51864 (local address 127.0.201.1:36515)
0619 12:09:57.217008 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.217010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.217021 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:09:57.217045 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.217046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.217056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.217111 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:57.217257 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.217752 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.218539 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.218734 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.218809 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.218967 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.219002 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.219056 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.219125 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.219142 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:09:57.220105 8621 raft_consensus.cc:1261] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Refusing update from remote peer 2fd1cdbc5ed44771ad2dba69d62d4c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.220134 8761 raft_consensus.cc:1261] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Refusing update from remote peer 2fd1cdbc5ed44771ad2dba69d62d4c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.220357 8884 consensus_queue.cc:1035] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.220464 8877 consensus_queue.cc:1035] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.223461 8323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:57.223563 8323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:57.224171 8323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Sending DeleteTablet for 3 replicas of tablet 51fce47dc7804b0cbc6624f733a2f34c
I20260619 12:09:57.224421 8741 tablet_service.cc:1514] Processing DeleteTablet for tablet 51fce47dc7804b0cbc6624f733a2f34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:53702
I20260619 12:09:57.224421 8601 tablet_service.cc:1514] Processing DeleteTablet for tablet 51fce47dc7804b0cbc6624f733a2f34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:33598
I20260619 12:09:57.224507 8859 tablet_replica.cc:330] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: stopping tablet replica
I20260619 12:09:57.224507 8860 tablet_replica.cc:330] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: stopping tablet replica
I20260619 12:09:57.224499 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet 51fce47dc7804b0cbc6624f733a2f34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:51848
I20260619 12:09:57.224551 8859 raft_consensus.cc:2229] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:57.224555 8860 raft_consensus.cc:2229] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:57.224579 8858 tablet_replica.cc:330] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: stopping tablet replica
I20260619 12:09:57.224589 8859 raft_consensus.cc:2258] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.224592 8860 raft_consensus.cc:2258] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.224622 8858 raft_consensus.cc:2229] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:57.224699 8858 raft_consensus.cc:2258] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.224969 8858 ts_tablet_manager.cc:1905] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.224975 8859 ts_tablet_manager.cc:1905] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.225040 8323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:09:57.225185 8323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:09:57.226341 8741 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea64bbea6234738ae5e9f632d6b8237 (DEFAULT_TABLE table=test-table [id=18ad46677f114020ac3a8f3f777d5f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:57.226528 8858 ts_tablet_manager.cc:1918] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:57.226550 8859 ts_tablet_manager.cc:1918] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:57.226585 8858 log.cc:1196] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/51fce47dc7804b0cbc6624f733a2f34c
I20260619 12:09:57.226600 8859 log.cc:1196] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/51fce47dc7804b0cbc6624f733a2f34c
I20260619 12:09:57.226768 8741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea64bbea6234738ae5e9f632d6b8237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.226806 8858 ts_tablet_manager.cc:1939] T 51fce47dc7804b0cbc6624f733a2f34c P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting consensus metadata
I20260619 12:09:57.226850 8859 ts_tablet_manager.cc:1939] T 51fce47dc7804b0cbc6624f733a2f34c P c5c11c3ca4174c878281f8851d5a12f2: Deleting consensus metadata
I20260619 12:09:57.227787 8835 tablet_bootstrap.cc:492] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Bootstrap starting.
I20260619 12:09:57.227919 8310 catalog_manager.cc:4841] TS c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333): tablet 51fce47dc7804b0cbc6624f733a2f34c (table test-table [id=5622e364cc7d402aa1996dd97400329c]) successfully deleted
I20260619 12:09:57.228179 8309 catalog_manager.cc:4841] TS 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515): tablet 51fce47dc7804b0cbc6624f733a2f34c (table test-table [id=5622e364cc7d402aa1996dd97400329c]) successfully deleted
I20260619 12:09:57.228323 8461 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea64bbea6234738ae5e9f632d6b8237 (DEFAULT_TABLE table=test-table [id=18ad46677f114020ac3a8f3f777d5f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:57.228343 8835 tablet_bootstrap.cc:654] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.228422 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea64bbea6234738ae5e9f632d6b8237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.228785 8601 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea64bbea6234738ae5e9f632d6b8237 (DEFAULT_TABLE table=test-table [id=18ad46677f114020ac3a8f3f777d5f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:09:57.228897 8601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea64bbea6234738ae5e9f632d6b8237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:57.229022 8835 tablet_bootstrap.cc:492] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: No bootstrap required, opened a new log
I20260619 12:09:57.229064 8835 ts_tablet_manager.cc:1397] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:09:57.229210 8835 raft_consensus.cc:348] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.229269 8835 raft_consensus.cc:374] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.229286 8835 raft_consensus.cc:729] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c11c3ca4174c878281f8851d5a12f2, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.229326 8835 consensus_queue.cc:260] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.229421 8835 ts_tablet_manager.cc:1428] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.229700 8837 tablet_bootstrap.cc:492] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Bootstrap starting.
I20260619 12:09:57.230021 8837 tablet_bootstrap.cc:654] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.230136 8836 tablet_bootstrap.cc:492] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Bootstrap starting.
I20260619 12:09:57.230417 8836 tablet_bootstrap.cc:654] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:57.230424 8860 ts_tablet_manager.cc:1905] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.230921 8836 tablet_bootstrap.cc:492] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: No bootstrap required, opened a new log
I20260619 12:09:57.230959 8836 ts_tablet_manager.cc:1397] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:09:57.231055 8836 raft_consensus.cc:348] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.231112 8836 raft_consensus.cc:374] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.231132 8836 raft_consensus.cc:729] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.231173 8836 consensus_queue.cc:260] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.231282 8836 ts_tablet_manager.cc:1428] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.231954 8860 ts_tablet_manager.cc:1918] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:09:57.232014 8860 log.cc:1196] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/51fce47dc7804b0cbc6624f733a2f34c
I20260619 12:09:57.232252 8860 ts_tablet_manager.cc:1939] T 51fce47dc7804b0cbc6624f733a2f34c P 6a790a214c144f23aca7512cc8f20256: Deleting consensus metadata
I20260619 12:09:57.232561 8309 catalog_manager.cc:4841] TS 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639): tablet 51fce47dc7804b0cbc6624f733a2f34c (table test-table [id=5622e364cc7d402aa1996dd97400329c]) successfully deleted
I20260619 12:09:57.232707 8837 tablet_bootstrap.cc:492] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: No bootstrap required, opened a new log
I20260619 12:09:57.232754 8837 ts_tablet_manager.cc:1397] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:09:57.232861 8837 raft_consensus.cc:348] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.232926 8837 raft_consensus.cc:374] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:57.232940 8837 raft_consensus.cc:729] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1cdbc5ed44771ad2dba69d62d4c18, State: Initialized, Role: FOLLOWER
I20260619 12:09:57.232990 8837 consensus_queue.cc:260] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.233080 8837 ts_tablet_manager.cc:1428] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:57.383050 8855 raft_consensus.cc:482] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:09:57.383172 8855 raft_consensus.cc:504] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.383383 8855 leader_election.cc:288] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:57.383602 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea64bbea6234738ae5e9f632d6b8237" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" is_pre_election: true
I20260619 12:09:57.383606 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea64bbea6234738ae5e9f632d6b8237" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2" is_pre_election: true
I20260619 12:09:57.383713 8481 raft_consensus.cc:2454] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 0.
I20260619 12:09:57.383713 8761 raft_consensus.cc:2454] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 0.
I20260619 12:09:57.383898 8553 leader_election.cc:302] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1cdbc5ed44771ad2dba69d62d4c18, 6a790a214c144f23aca7512cc8f20256; no voters:
I20260619 12:09:57.384008 8855 raft_consensus.cc:2781] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:09:57.384047 8855 raft_consensus.cc:482] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:09:57.384078 8855 raft_consensus.cc:3037] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.384774 8855 raft_consensus.cc:504] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.384922 8855 leader_election.cc:288] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 election: Requested vote from peers 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515), c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333)
I20260619 12:09:57.385076 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea64bbea6234738ae5e9f632d6b8237" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c11c3ca4174c878281f8851d5a12f2"
I20260619 12:09:57.385113 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea64bbea6234738ae5e9f632d6b8237" candidate_uuid: "6a790a214c144f23aca7512cc8f20256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18"
I20260619 12:09:57.385150 8761 raft_consensus.cc:3037] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.385187 8481 raft_consensus.cc:3037] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:57.385749 8761 raft_consensus.cc:2454] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 1.
I20260619 12:09:57.385757 8481 raft_consensus.cc:2454] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a790a214c144f23aca7512cc8f20256 in term 1.
I20260619 12:09:57.386000 8554 leader_election.cc:302] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a790a214c144f23aca7512cc8f20256, c5c11c3ca4174c878281f8851d5a12f2; no voters:
I20260619 12:09:57.386139 8855 raft_consensus.cc:2781] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:57.386235 8855 raft_consensus.cc:686] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 1 LEADER]: Becoming Leader. State: Replica: 6a790a214c144f23aca7512cc8f20256, State: Running, Role: LEADER
I20260619 12:09:57.386291 8855 consensus_queue.cc:237] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } }
I20260619 12:09:57.386853 8324 catalog_manager.cc:5495] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a790a214c144f23aca7512cc8f20256 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "6a790a214c144f23aca7512cc8f20256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a790a214c144f23aca7512cc8f20256" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 } health_report { overall_health: UNKNOWN } } }
I20260619 12:09:57.417358 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.415129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33614 (local address 127.0.201.2:43639)
0619 12:09:57.415179 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:57.415181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:57.415213 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:09:57.415238 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:57.415239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:57.415253 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:57.415302 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:09:57.415418 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.415893 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.416651 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:57.416855 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:57.416929 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:09:57.417086 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:09:57.417121 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:09:57.417189 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:09:57.417245 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 12:09:57.417264 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:09:57.419763 8481 raft_consensus.cc:1261] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Refusing update from remote peer 6a790a214c144f23aca7512cc8f20256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.419904 8761 raft_consensus.cc:1261] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Refusing update from remote peer 6a790a214c144f23aca7512cc8f20256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:09:57.419973 8855 consensus_queue.cc:1035] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd1cdbc5ed44771ad2dba69d62d4c18" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.420100 8855 consensus_queue.cc:1035] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c11c3ca4174c878281f8851d5a12f2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:09:57.430903 8324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:57.431020 8324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:09:57.431630 8324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0e3edfad7e44fc48cb895d326c652e9: Sending DeleteTablet for 3 replicas of tablet 2ea64bbea6234738ae5e9f632d6b8237
I20260619 12:09:57.432094 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea64bbea6234738ae5e9f632d6b8237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:51848
I20260619 12:09:57.432096 8601 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea64bbea6234738ae5e9f632d6b8237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:33598
I20260619 12:09:57.432194 8860 tablet_replica.cc:330] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: stopping tablet replica
I20260619 12:09:57.432219 8858 tablet_replica.cc:330] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: stopping tablet replica
I20260619 12:09:57.432241 8860 raft_consensus.cc:2229] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:09:57.432271 8858 raft_consensus.cc:2229] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:57.432299 8860 raft_consensus.cc:2258] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.432313 8858 raft_consensus.cc:2258] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.432663 8860 ts_tablet_manager.cc:1905] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.433488 8741 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea64bbea6234738ae5e9f632d6b8237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:09:57 UTC) from {username='oryx', principal='oryx/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:53702
I20260619 12:09:57.434239 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 8396
I20260619 12:09:57.434294 8859 tablet_replica.cc:330] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: stopping tablet replica
I20260619 12:09:57.434367 8859 raft_consensus.cc:2229] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:09:57.434401 8859 raft_consensus.cc:2258] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:09:57.434406 8858 ts_tablet_manager.cc:1905] T 2ea64bbea6234738ae5e9f632d6b8237 P 2fd1cdbc5ed44771ad2dba69d62d4c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.435783 8859 ts_tablet_manager.cc:1905] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:09:57.436187 8860 ts_tablet_manager.cc:1918] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:57.436244 8860 log.cc:1196] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/2ea64bbea6234738ae5e9f632d6b8237
I20260619 12:09:57.436473 8860 ts_tablet_manager.cc:1939] T 2ea64bbea6234738ae5e9f632d6b8237 P 6a790a214c144f23aca7512cc8f20256: Deleting consensus metadata
I20260619 12:09:57.436834 8309 catalog_manager.cc:4841] TS 6a790a214c144f23aca7512cc8f20256 (127.0.201.2:43639): tablet 2ea64bbea6234738ae5e9f632d6b8237 (table test-table [id=18ad46677f114020ac3a8f3f777d5f0e]) successfully deleted
I20260619 12:09:57.437175 8859 ts_tablet_manager.cc:1918] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:09:57.437220 8859 log.cc:1196] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/2ea64bbea6234738ae5e9f632d6b8237
I20260619 12:09:57.437546 8859 ts_tablet_manager.cc:1939] T 2ea64bbea6234738ae5e9f632d6b8237 P c5c11c3ca4174c878281f8851d5a12f2: Deleting consensus metadata
I20260619 12:09:57.437899 8310 catalog_manager.cc:4841] TS c5c11c3ca4174c878281f8851d5a12f2 (127.0.201.3:45333): tablet 2ea64bbea6234738ae5e9f632d6b8237 (table test-table [id=18ad46677f114020ac3a8f3f777d5f0e]) successfully deleted
W20260619 12:09:57.443822 8309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:36515 (error 108)
I20260619 12:09:57.444085 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 8536
I20260619 12:09:57.444432 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:57.444229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36515 (local address 127.0.0.1:51870)
0619 12:09:57.444347 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:09:57.444385 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
W20260619 12:09:57.444506 8309 catalog_manager.cc:4584] TS 2fd1cdbc5ed44771ad2dba69d62d4c18 (127.0.201.1:36515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ea64bbea6234738ae5e9f632d6b8237: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36515: connect: Connection refused (error 111)
I20260619 12:09:57.449807 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 8676
I20260619 12:09:57.454739 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 8292
2026-06-19T12:09:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2815 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8922](info): set up 2 sockets
Jun 19 12:09:57 dist-test-slave-c910 krb5kdc[8922](info): commencing operation
krb5kdc: starting...
W20260619 12:09:58.487819 804 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:58 dist-test-slave-c910 krb5kdc[8922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:58.604233 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:37535
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:37535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:58.683447 8935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:58.683666 8935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:58.683691 8935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:58.684867 8935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:58.684908 8935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:58.684923 8935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:58.684935 8935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:58.684947 8935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:58.686460 8935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:37535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:37535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:58.686710 8935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:58.686961 8935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:58.691696 8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:58.691663 8941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:58.691838 8940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:58.692021 8935 server_base.cc:1034] running on GCE node
I20260619 12:09:58.692653 8935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:58.692965 8935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:58.694103 8935 hybrid_clock.cc:648] HybridClock initialized: now 1781870998694094 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T12:09:58Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1246 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:09:58 dist-test-slave-c910 krb5kdc[8948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:09:58 dist-test-slave-c910 krb5kdc[8948](info): set up 2 sockets
Jun 19 12:09:58 dist-test-slave-c910 krb5kdc[8948](info): commencing operation
krb5kdc: starting...
W20260619 12:09:59.743736 804 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:09:59 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:09:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:09:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:09:59.857532 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36255
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:09:59.938246 8961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:09:59.938457 8961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:09:59.938481 8961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:09:59.939584 8961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:09:59.939622 8961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:09:59.939639 8961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:09:59.939654 8961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:09:59.939668 8961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:09:59.941105 8961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:36255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:09:59.941298 8961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:09:59.941469 8961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:09:59.946238 8967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:59.946385 8961 server_base.cc:1034] running on GCE node
W20260619 12:09:59.946274 8966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:09:59.946521 8969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:09:59.946787 8961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:09:59.947127 8961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:09:59.948297 8961 hybrid_clock.cc:648] HybridClock initialized: now 1781870999948284 us; error 40 us; skew 500 ppm
Jun 19 12:09:59 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:09:59.954345 8961 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:09:59.954818 8961 webserver.cc:460] Webserver started at http://127.0.201.62:37743/ using document root <none> and password file <none>
I20260619 12:09:59.955053 8961 fs_manager.cc:359] Metadata directory not provided
I20260619 12:09:59.955127 8961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:09:59.955299 8961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:09:59.956369 8961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "f7c2936b0b674660a4bb777d438d0583"
format_stamp: "Formatted at 2026-06-19 12:09:59 on dist-test-slave-c910"
server_key: "891ed34400f78dd43518df1bd0d1b61f"
server_key_iv: "47214222ff8a2a755e9f99883aa7d3cd"
server_key_version: "encryptionkey@0"
I20260619 12:09:59.956723 8961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "f7c2936b0b674660a4bb777d438d0583"
format_stamp: "Formatted at 2026-06-19 12:09:59 on dist-test-slave-c910"
server_key: "891ed34400f78dd43518df1bd0d1b61f"
server_key_iv: "47214222ff8a2a755e9f99883aa7d3cd"
server_key_version: "encryptionkey@0"
I20260619 12:09:59.957991 8961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:09:59.958709 8976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:09:59.958877 8961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:09:59.958985 8961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "f7c2936b0b674660a4bb777d438d0583"
format_stamp: "Formatted at 2026-06-19 12:09:59 on dist-test-slave-c910"
server_key: "891ed34400f78dd43518df1bd0d1b61f"
server_key_iv: "47214222ff8a2a755e9f99883aa7d3cd"
server_key_version: "encryptionkey@0"
I20260619 12:09:59.959064 8961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:09:59.975565 8961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:09:59.977378 8961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:09:59.977540 8961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:09:59.981527 8961 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:36255
I20260619 12:09:59.981559 9028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:36255 every 8 connection(s)
I20260619 12:09:59.982024 8961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:09:59.982632 9029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:09:59.984881 9029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Bootstrap starting.
I20260619 12:09:59.985458 9029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Neither blocks nor log segments found. Creating new log.
I20260619 12:09:59.985747 9029 log.cc:824] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Log is configured to *not* fsync() on all Append() calls
I20260619 12:09:59.986404 9029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: No bootstrap required, opened a new log
I20260619 12:09:59.987969 9029 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:09:59.988099 9029 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:09:59.988122 9029 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c2936b0b674660a4bb777d438d0583, State: Initialized, Role: FOLLOWER
I20260619 12:09:59.988288 9029 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:09:59.988350 9029 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:09:59.988394 9029 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:09:59.988440 9029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:09:59.989308 9029 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:09:59.989392 9029 leader_election.cc:302] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7c2936b0b674660a4bb777d438d0583; no voters:
I20260619 12:09:59.989557 9029 leader_election.cc:288] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:09:59.989666 9032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:09:59.989856 9032 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 LEADER]: Becoming Leader. State: Replica: f7c2936b0b674660a4bb777d438d0583, State: Running, Role: LEADER
I20260619 12:09:59.989874 9029 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:09:59.989941 9032 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:09:59.990350 9034 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7c2936b0b674660a4bb777d438d0583. Latest consensus state: current_term: 1 leader_uuid: "f7c2936b0b674660a4bb777d438d0583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } } }
I20260619 12:09:59.990443 9034 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:59.990628 9033 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7c2936b0b674660a4bb777d438d0583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } } }
I20260619 12:09:59.990870 9033 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: This master's current role is: LEADER
I20260619 12:09:59.991029 9041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:09:59.991546 9041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:09:59.992163 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 8961
I20260619 12:09:59.992273 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:09:59.992460 804 external_mini_cluster.cc:1428] Setting key a334f96e2adda7fe1f32f531fafb9c35
I20260619 12:09:59.993520 9041 catalog_manager.cc:1347] Generated new cluster ID: 2bbbd9b25fd142979ff4bd14a5f2b597
I20260619 12:09:59.993575 9041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:09:59 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781870999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:00.004808 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:09:59.992947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.0.201.62:36255)
0619 12:09:59.993193 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 12:09:59.993202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:09:59.993238 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:09:59.993331 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:09:59.993336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:09:59.993376 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:09:59.995307 (+ 1931us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:09:59.995397 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:59.996770 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:59.997897 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:09:59.998506 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:09:59.998603 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.001327 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:00.001331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:00.001331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:00.001343 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:00.003300 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.003654 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.003657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.003657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.003759 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.003941 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.003942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.003942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.004240 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:00.004384 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:00.004456 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 12:10:00.004585 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":71,"threads_started":1}
I20260619 12:10:00.019582 9041 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:10:00.020135 9041 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:10:00.024134 9041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Generated new TSK 0
I20260619 12:10:00.024325 9041 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:10:00.051879 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36255
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:00.134222 9055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:00.134471 9055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:00.134500 9055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:00.135798 9055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:00.135843 9055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:00.135872 9055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:10:00.137404 9055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:00.137651 9055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:00.137915 9055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:00.142962 9061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.143038 9055 server_base.cc:1034] running on GCE node
W20260619 12:10:00.143112 9063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.143148 9060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.143394 9055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:00.143663 9055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:00.144829 9055 hybrid_clock.cc:648] HybridClock initialized: now 1781871000144814 us; error 43 us; skew 500 ppm
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:00.147432 9055 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:10:00.147876 9055 webserver.cc:460] Webserver started at http://127.0.201.1:43481/ using document root <none> and password file <none>
I20260619 12:10:00.148116 9055 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:00.148185 9055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:00.148339 9055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:00.149415 9055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "3a33b13183f44d488aceb72d51ad7b13"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "882d6e62b8664fab033a3254626f2f5b"
server_key_iv: "075020a94f76f12f443a5195954fdd2c"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.149782 9055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "3a33b13183f44d488aceb72d51ad7b13"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "882d6e62b8664fab033a3254626f2f5b"
server_key_iv: "075020a94f76f12f443a5195954fdd2c"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.151093 9055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:10:00.151932 9070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.152114 9055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:10:00.152208 9055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "3a33b13183f44d488aceb72d51ad7b13"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "882d6e62b8664fab033a3254626f2f5b"
server_key_iv: "075020a94f76f12f443a5195954fdd2c"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.152295 9055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:00.190043 9055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:00.192204 9055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:00.192373 9055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:00.192617 9055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:00.192926 9055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:00.192981 9055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.193025 9055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:00.193053 9055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.198570 9055 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:45463
I20260619 12:10:00.198619 9183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:45463 every 8 connection(s)
I20260619 12:10:00.198958 9055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:00.206740 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9055
I20260619 12:10:00.206831 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:10:00.207053 804 external_mini_cluster.cc:1428] Setting key a2074448924c65812910187e48450571
I20260619 12:10:00.208601 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.199762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:53671 (local address 127.0.201.62:36255)
0619 12:10:00.199830 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:00.199832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:00.200025 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:10:00.200957 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:00.200958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:00.200969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:00.201027 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:00.201603 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.202097 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.203029 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.203249 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.203303 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.205261 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:00.205262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:00.205263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:00.205269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:00.207261 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.207663 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.207665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.207665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.207724 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.207968 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.207969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.207969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.208088 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:00.208165 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:00.208416 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 12:10:00.208498 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:10:00.208799 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.199748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:53671)
0619 12:10:00.199903 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:00.199942 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:00.200835 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:00.201090 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:00.201094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:00.201330 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:00.201511 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.201515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.202181 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.202181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.202953 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.202954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.203308 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.203308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.203399 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.203803 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:00.203810 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:00.205175 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:00.207351 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:00.207357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:00.207359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:00.207572 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:00.207804 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:00.207804 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:00.207805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:00.207902 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:00.208156 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:00.208162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:00.208290 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 12:10:00.208396 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:10:00.208542 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260619 12:10:00.209412 9184 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36255
I20260619 12:10:00.209515 9184 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:00.209683 9184 heartbeater.cc:511] Master 127.0.201.62:36255 requested a full tablet report, sending...
I20260619 12:10:00.210211 8993 ts_manager.cc:194] Registered new tserver with Master: 3a33b13183f44d488aceb72d51ad7b13 (127.0.201.1:45463)
I20260619 12:10:00.210810 8993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:53671
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:10:00.230304 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36255
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:00.309356 9189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:00.309551 9189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:00.309576 9189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:00.310696 9189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:00.310745 9189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:00.310772 9189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:10:00.312292 9189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:00.312520 9189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:00.312750 9189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:00.317719 9194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.317898 9189 server_base.cc:1034] running on GCE node
W20260619 12:10:00.317716 9197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.317745 9195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.318164 9189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:00.318410 9189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:00.319649 9189 hybrid_clock.cc:648] HybridClock initialized: now 1781871000319645 us; error 33 us; skew 500 ppm
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:00.322255 9189 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:10:00.322696 9189 webserver.cc:460] Webserver started at http://127.0.201.2:43675/ using document root <none> and password file <none>
I20260619 12:10:00.322873 9189 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:00.322911 9189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:00.323000 9189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:00.324035 9189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "d2eb1a2b888047a8ba90f0ff5fbda114"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "9435312d5de8242b9d52109d1de63b2f"
server_key_iv: "c47614255e637c5d8c8ce605d7aedad8"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.324409 9189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "d2eb1a2b888047a8ba90f0ff5fbda114"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "9435312d5de8242b9d52109d1de63b2f"
server_key_iv: "c47614255e637c5d8c8ce605d7aedad8"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.325650 9189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:10:00.326320 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.326493 9189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:10:00.326570 9189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "d2eb1a2b888047a8ba90f0ff5fbda114"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "9435312d5de8242b9d52109d1de63b2f"
server_key_iv: "c47614255e637c5d8c8ce605d7aedad8"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.326619 9189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:00.339790 9189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:00.341568 9189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:00.341673 9189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:00.341846 9189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:00.342077 9189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:00.342106 9189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.342125 9189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:00.342139 9189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.347756 9189 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44971
I20260619 12:10:00.347844 9317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44971 every 8 connection(s)
I20260619 12:10:00.348119 9189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:00.354708 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9189
I20260619 12:10:00.354821 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:10:00.355036 804 external_mini_cluster.cc:1428] Setting key be1f1b0777c20e01b7783ab737cc1105
I20260619 12:10:00.357206 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.348781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:35067 (local address 127.0.201.62:36255)
0619 12:10:00.348820 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:00.348822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:00.349009 (+ 187us) server_negotiation.cc:407] Connection header received
0619 12:10:00.349891 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:00.349891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:00.349900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:00.349958 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:00.350510 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.351022 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.352016 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.352226 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.352282 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.354195 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:00.354196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:00.354196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:00.354203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:00.355841 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.356290 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.356291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.356292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.356345 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.356592 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.356593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.356593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.356704 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:00.356779 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:00.357034 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 12:10:00.357109 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:10:00.357384 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.348730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:35067)
0619 12:10:00.348888 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:00.348926 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:00.349809 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:00.350025 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:00.350029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:00.350271 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:00.350442 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.350446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.351096 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.351096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.351944 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.351946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.352299 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.352300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.352390 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.352807 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:00.352814 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:00.354081 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:00.355932 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:00.355938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:00.355940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:00.356202 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:00.356418 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:00.356419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:00.356419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:00.356525 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:00.356770 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:00.356775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:00.356909 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 12:10:00.357015 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:10:00.357162 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260619 12:10:00.357786 9318 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36255
I20260619 12:10:00.357896 9318 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:00.358057 9318 heartbeater.cc:511] Master 127.0.201.62:36255 requested a full tablet report, sending...
I20260619 12:10:00.358454 8993 ts_manager.cc:194] Registered new tserver with Master: d2eb1a2b888047a8ba90f0ff5fbda114 (127.0.201.2:44971)
I20260619 12:10:00.358952 8993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:35067
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:10:00.380064 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36255
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:00.460702 9323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:00.460887 9323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:00.460909 9323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:00.462109 9323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:00.462147 9323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:00.462172 9323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:10:00.463716 9323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:00.463927 9323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:00.464146 9323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:10:00.469100 9323 server_base.cc:1034] running on GCE node
W20260619 12:10:00.469053 9331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.469053 9329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.469062 9328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.469496 9323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:00.469676 9323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:00.470808 9323 hybrid_clock.cc:648] HybridClock initialized: now 1781871000470784 us; error 41 us; skew 500 ppm
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:00.473378 9323 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:10:00.473858 9323 webserver.cc:460] Webserver started at http://127.0.201.3:43275/ using document root <none> and password file <none>
I20260619 12:10:00.474088 9323 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:00.474149 9323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:00.474275 9323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:00.475414 9323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "389eafcaad7b4f8ea7a8c41cd32df248"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "1aec7cb482efdb7a357336c62fb0fb9e"
server_key_iv: "7e7d117e456e278de932632772cf04fe"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.475729 9323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "389eafcaad7b4f8ea7a8c41cd32df248"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "1aec7cb482efdb7a357336c62fb0fb9e"
server_key_iv: "7e7d117e456e278de932632772cf04fe"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.476975 9323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:10:00.477672 9338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.477854 9323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.477918 9323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "389eafcaad7b4f8ea7a8c41cd32df248"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "1aec7cb482efdb7a357336c62fb0fb9e"
server_key_iv: "7e7d117e456e278de932632772cf04fe"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.478003 9323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:00.499591 9323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:00.501379 9323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:00.501528 9323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:00.501741 9323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:00.502034 9323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:00.502068 9323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.502089 9323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:00.502101 9323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.507915 9323 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:40189
I20260619 12:10:00.507956 9451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:40189 every 8 connection(s)
I20260619 12:10:00.508244 9323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:00.514662 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9323
I20260619 12:10:00.514762 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:10:00.514921 804 external_mini_cluster.cc:1428] Setting key 30c6569ea8c5f1501f591cec059ad1b4
I20260619 12:10:00.517139 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.508972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:50885 (local address 127.0.201.62:36255)
0619 12:10:00.509017 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:00.509019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:00.509187 (+ 168us) server_negotiation.cc:407] Connection header received
0619 12:10:00.510103 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:00.510105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:00.510116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:00.510190 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:00.510743 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.511242 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.512158 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.512339 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.512393 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.514318 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:00.514320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:00.514320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:00.514326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:00.516022 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.516379 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.516380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.516380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.516426 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:00.516649 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:00.516649 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:00.516649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:00.516741 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:00.516801 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:00.517022 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:10:00.517078 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:10:00.517310 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.508924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:50885)
0619 12:10:00.509073 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:00.509113 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:00.509983 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:00.510243 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:00.510247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:00.510477 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:00.510652 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.510655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.511309 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.511310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.512083 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.512085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.512409 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.512410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.512497 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.512912 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:00.512919 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:00.514226 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:00.516090 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:00.516095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:00.516097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:00.516315 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:00.516484 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:00.516484 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:00.516484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:00.516574 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:00.516808 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:00.516812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:00.516916 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 12:10:00.516995 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:10:00.517102 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260619 12:10:00.517768 9452 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36255
I20260619 12:10:00.517843 9452 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:00.518023 9452 heartbeater.cc:511] Master 127.0.201.62:36255 requested a full tablet report, sending...
I20260619 12:10:00.518378 8993 ts_manager.cc:194] Registered new tserver with Master: 389eafcaad7b4f8ea7a8c41cd32df248 (127.0.201.3:40189)
I20260619 12:10:00.518695 8993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:50885
I20260619 12:10:00.526479 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab.
Entry for principal oryx/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab.
WARNING: no policy specified for oryx/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab.
Entry for principal oryx/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab.
WARNING: no policy specified for oryx/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab.
Entry for principal oryx/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab.
I20260619 12:10:00.587528 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9055
I20260619 12:10:00.592000 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9189
I20260619 12:10:00.595988 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9323
I20260619 12:10:00.600150 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 8961
I20260619 12:10:00.604869 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36255
--webserver_interface=127.0.201.62
--webserver_port=37743
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:00.683841 9461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:00.684041 9461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:00.684062 9461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:00.685205 9461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:10:00.685246 9461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:00.685261 9461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:00.685273 9461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:10:00.685285 9461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:10:00.686757 9461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:36255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=37743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:00.686975 9461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:00.687175 9461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:00.691761 9469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.691810 9467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.691964 9461 server_base.cc:1034] running on GCE node
W20260619 12:10:00.691763 9466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.692322 9461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:00.692574 9461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:00.693733 9461 hybrid_clock.cc:648] HybridClock initialized: now 1781871000693722 us; error 31 us; skew 500 ppm
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:00.696218 9461 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:10:00.696626 9461 webserver.cc:460] Webserver started at http://127.0.201.62:37743/ using document root <none> and password file <none>
I20260619 12:10:00.696858 9461 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:00.696938 9461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:00.698232 9461 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:00.698854 9476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.699021 9461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:00.699120 9461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "f7c2936b0b674660a4bb777d438d0583"
format_stamp: "Formatted at 2026-06-19 12:09:59 on dist-test-slave-c910"
server_key: "891ed34400f78dd43518df1bd0d1b61f"
server_key_iv: "47214222ff8a2a755e9f99883aa7d3cd"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.699467 9461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:00.734280 9461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:00.736233 9461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:00.736388 9461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:00.740080 9461 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:36255
I20260619 12:10:00.740119 9528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:36255 every 8 connection(s)
I20260619 12:10:00.740617 9461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:10:00.742069 9529 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:10:00.742791 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Bootstrap starting.
I20260619 12:10:00.749480 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9461
I20260619 12:10:00.749887 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:45463
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:36255
--webserver_port=43481
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:10:00.752110 9529 log.cc:824] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:00.754093 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:10:00.754443 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Bootstrap complete.
I20260619 12:10:00.756888 9529 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:10:00.757162 9529 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c2936b0b674660a4bb777d438d0583, State: Initialized, Role: FOLLOWER
I20260619 12:10:00.757361 9529 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:10:00.757432 9529 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:10:00.757488 9529 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:10:00.757553 9529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:10:00.758635 9529 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:10:00.758730 9529 leader_election.cc:302] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7c2936b0b674660a4bb777d438d0583; no voters:
I20260619 12:10:00.758967 9529 leader_election.cc:288] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:10:00.759003 9533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:10:00.759177 9533 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [term 2 LEADER]: Becoming Leader. State: Replica: f7c2936b0b674660a4bb777d438d0583, State: Running, Role: LEADER
I20260619 12:10:00.759269 9529 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:10:00.759392 9533 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } }
I20260619 12:10:00.759866 9535 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7c2936b0b674660a4bb777d438d0583. Latest consensus state: current_term: 2 leader_uuid: "f7c2936b0b674660a4bb777d438d0583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } } }
I20260619 12:10:00.759876 9534 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f7c2936b0b674660a4bb777d438d0583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c2936b0b674660a4bb777d438d0583" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36255 } } }
I20260619 12:10:00.759996 9534 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:00.759989 9535 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:00.760507 9542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:10:00.761106 9542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:10:00.761336 9542 catalog_manager.cc:1259] Loaded cluster ID: 2bbbd9b25fd142979ff4bd14a5f2b597
I20260619 12:10:00.761534 9542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:10:00.762424 9542 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:10:00.762642 9542 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f7c2936b0b674660a4bb777d438d0583: Loaded TSK: 0
I20260619 12:10:00.762885 9542 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:10:00.841897 9531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:00.842118 9531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:00.842155 9531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:00.843317 9531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:00.843372 9531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:00.843416 9531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:10:00.844786 9531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.1.keytab
--principal=oryx/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:45463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=43481
--tserver_master_addrs=127.0.201.62:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:00.845031 9531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:00.845283 9531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:00.850819 9556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.850903 9558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.851099 9531 server_base.cc:1034] running on GCE node
W20260619 12:10:00.850821 9555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.851358 9531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:00.851598 9531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:00.852789 9531 hybrid_clock.cc:648] HybridClock initialized: now 1781871000852767 us; error 37 us; skew 500 ppm
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:00.855419 9531 init.cc:377] Logged in from keytab as oryx/127.0.201.1@KRBTEST.COM (short username oryx)
I20260619 12:10:00.855899 9531 webserver.cc:460] Webserver started at http://127.0.201.1:43481/ using document root <none> and password file <none>
I20260619 12:10:00.856150 9531 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:00.856207 9531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:00.857518 9531 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:10:00.858147 9565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.858290 9531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:10:00.858373 9531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "3a33b13183f44d488aceb72d51ad7b13"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "882d6e62b8664fab033a3254626f2f5b"
server_key_iv: "075020a94f76f12f443a5195954fdd2c"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.858726 9531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:00.884719 9531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:00.886924 9531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:00.887068 9531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:00.887357 9531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:00.887742 9531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:00.887778 9531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.887800 9531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:00.887813 9531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.895013 9531 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:45463
I20260619 12:10:00.895089 9678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:45463 every 8 connection(s)
I20260619 12:10:00.895434 9531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:10:00.895627 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9531
I20260619 12:10:00.895936 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:44971
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:36255
--webserver_port=43675
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:00.905092 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.896158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:46141)
0619 12:10:00.896319 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:00.896353 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:00.896911 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:00.897382 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:00.897386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:00.898191 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:00.898471 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.898477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.900181 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.900182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.900945 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.900947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.901834 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.901835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.902009 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.902534 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:00.902542 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:00.904956 (+ 2414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
W20260619 12:10:00.905323 9679 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:36255 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.201.62:36255: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
W20260619 12:10:00.905660 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:00.896510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:46141 (local address 127.0.201.62:36255)
0619 12:10:00.896698 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:00.896707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:00.896743 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:10:00.897102 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:00.897108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:00.897166 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:00.897317 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:00.898620 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.900088 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.901039 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.901730 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.901843 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.905135 (+ 3292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:46141: BlockingRecv error: recv got EOF from 127.0.201.1:46141 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:00.910418 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.905554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:38567)
0619 12:10:00.905835 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:00.905847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:00.905904 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:00.906059 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:00.906060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:00.906228 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:00.906288 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.906290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.906909 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.906910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.907753 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.907755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.908100 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.908101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.908165 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.908557 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:00.908562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:00.910372 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265}
W20260619 12:10:00.910583 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:00.905653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:38567 (local address 127.0.201.62:36255)
0619 12:10:00.905755 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:00.905757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:00.905908 (+ 151us) server_negotiation.cc:407] Connection header received
0619 12:10:00.905973 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:00.905973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:00.905983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:00.906033 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:00.906368 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.906833 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.907837 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.908023 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.908079 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.910469 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:38567: BlockingRecv error: recv got EOF from 127.0.201.1:38567 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":93,"threads_started":1}
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:00.914949 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:00.910668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:34147)
0619 12:10:00.910800 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:00.910808 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:00.910839 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:00.910972 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:00.910972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:00.911071 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:00.911098 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.911099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.911971 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.911972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.912436 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:00.912438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.912902 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.912902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:00.912938 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.913349 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:00.913353 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:00.914884 (+ 1531us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260619 12:10:00.915037 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:00.910764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:34147 (local address 127.0.201.62:36255)
0619 12:10:00.910816 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:00.910818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:00.910829 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:10:00.910893 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:00.910894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:00.910904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:00.910961 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:00.911158 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.911896 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.912507 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:00.912821 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:00.912900 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:00.914961 (+ 2061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:34147: BlockingRecv error: recv got EOF from 127.0.201.1:34147 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 12:10:00.915082 9679 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:10:00.977775 9683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:00.977982 9683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:00.978004 9683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:00.979061 9683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:00.979094 9683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:00.979117 9683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:10:00.980415 9683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.2.keytab
--principal=oryx/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:44971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=43675
--tserver_master_addrs=127.0.201.62:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:00.980589 9683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:00.980746 9683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:00.985662 9690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.985673 9689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:00.985668 9692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:00.985914 9683 server_base.cc:1034] running on GCE node
I20260619 12:10:00.986160 9683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:00.986405 9683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:00.987560 9683 hybrid_clock.cc:648] HybridClock initialized: now 1781871000987544 us; error 25 us; skew 500 ppm
Jun 19 12:10:00 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871000, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:00.989938 9683 init.cc:377] Logged in from keytab as oryx/127.0.201.2@KRBTEST.COM (short username oryx)
I20260619 12:10:00.990342 9683 webserver.cc:460] Webserver started at http://127.0.201.2:43675/ using document root <none> and password file <none>
I20260619 12:10:00.990563 9683 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:00.990641 9683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:00.992038 9683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:00.992609 9699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:00.992767 9683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:00.992861 9683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "d2eb1a2b888047a8ba90f0ff5fbda114"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "9435312d5de8242b9d52109d1de63b2f"
server_key_iv: "c47614255e637c5d8c8ce605d7aedad8"
server_key_version: "encryptionkey@0"
I20260619 12:10:00.993204 9683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:01.004103 9683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:01.005955 9683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:01.006104 9683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:01.006315 9683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:01.006597 9683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:01.006649 9683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:01.006685 9683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:01.006713 9683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:01.012398 9683 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44971
I20260619 12:10:01.012483 9812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44971 every 8 connection(s)
I20260619 12:10:01.012770 9683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:10:01.019366 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.013387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:42695 (local address 127.0.201.62:36255)
0619 12:10:01.013462 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.013464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.013647 (+ 183us) server_negotiation.cc:407] Connection header received
0619 12:10:01.014077 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.014078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.014092 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.014141 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.014969 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.015502 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.016411 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.016590 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.016637 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.019321 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:42695: BlockingRecv error: recv got EOF from 127.0.201.2:42695 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:10:01.019369 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.013374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:42695)
0619 12:10:01.013539 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.013570 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.014002 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.014188 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.014191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.014728 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.014899 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.014904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.015573 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.015574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.016344 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.016347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.016653 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.016654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.016739 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.017226 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.017242 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.019181 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
W20260619 12:10:01.019618 9813 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:36255 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.201.62:36255: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
I20260619 12:10:01.021404 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9683
I20260619 12:10:01.021695 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:40189
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:36255
--webserver_port=43275
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:01.024117 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.019809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:36077)
0619 12:10:01.019853 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.019863 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.019919 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.020059 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.020060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.020177 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.020215 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.020216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.020810 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.020811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.021394 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.021396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.021805 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.021806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.021876 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.022217 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.022222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.024077 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:10:01.024262 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.019875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:36077 (local address 127.0.201.62:36255)
0619 12:10:01.019932 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.019934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.019942 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:10:01.019967 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.019969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.019977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.020054 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.020295 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.020731 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.021500 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.021700 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.021780 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.024183 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:36077: BlockingRecv error: recv got EOF from 127.0.201.2:36077 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:01.028709 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.024527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:58833)
0619 12:10:01.024572 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.024583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.024626 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.024743 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.024743 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.024862 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.024883 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.024884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.025604 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.025605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.026075 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.026076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.026434 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.026435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.026483 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.026911 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.026914 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.028638 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 12:10:01.028777 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.024595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:58833 (local address 127.0.201.62:36255)
0619 12:10:01.024647 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.024649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.024656 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:10:01.024676 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.024677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.024685 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.024746 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.024958 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.025505 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.026150 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.026355 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.026432 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.028704 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:58833: BlockingRecv error: recv got EOF from 127.0.201.2:58833 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 12:10:01.028988 9813 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:10:01.101408 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:01.101671 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:01.101709 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:01.102880 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:01.102952 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:01.102994 9816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:10:01.104372 9816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/krb5kdc/oryx_127.0.201.3.keytab
--principal=oryx/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:40189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=43275
--tserver_master_addrs=127.0.201.62:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:01.104631 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:01.104852 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:10:01.109648 9816 server_base.cc:1034] running on GCE node
W20260619 12:10:01.109651 9821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:01.109632 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:01.109627 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:01.110136 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:01.110400 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:01.111564 9816 hybrid_clock.cc:648] HybridClock initialized: now 1781871001111552 us; error 26 us; skew 500 ppm
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871001, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:01.113948 9816 init.cc:377] Logged in from keytab as oryx/127.0.201.3@KRBTEST.COM (short username oryx)
I20260619 12:10:01.114358 9816 webserver.cc:460] Webserver started at http://127.0.201.3:43275/ using document root <none> and password file <none>
I20260619 12:10:01.114575 9816 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:01.114655 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:01.115975 9816 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:01.116557 9831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:01.116712 9816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:01.116806 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "389eafcaad7b4f8ea7a8c41cd32df248"
format_stamp: "Formatted at 2026-06-19 12:10:00 on dist-test-slave-c910"
server_key: "1aec7cb482efdb7a357336c62fb0fb9e"
server_key_iv: "7e7d117e456e278de932632772cf04fe"
server_key_version: "encryptionkey@0"
I20260619 12:10:01.117117 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:01.131620 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:01.133800 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:01.133952 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:01.134168 9816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:01.134469 9816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:01.134521 9816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:01.134560 9816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:01.134589 9816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:01.140127 9816 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:40189
I20260619 12:10:01.140161 9944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:40189 every 8 connection(s)
I20260619 12:10:01.140471 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:01.146916 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 9816
W20260619 12:10:01.147158 9682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.141155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:42407 (local address 127.0.201.62:36255)
0619 12:10:01.141201 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.141202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.141412 (+ 210us) server_negotiation.cc:407] Connection header received
0619 12:10:01.141859 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.141860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.141868 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.141925 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.142805 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.143295 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.144232 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.144416 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.144465 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.147105 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:42407: BlockingRecv error: recv got EOF from 127.0.201.3:42407 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:10:01.147231 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.141130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:42407)
0619 12:10:01.141288 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.141328 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.141781 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.141996 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.141999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.142520 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.142727 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.142732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.143368 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.143369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.144160 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.144162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.144481 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.144482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.144595 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.145064 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.145073 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.147031 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
W20260619 12:10:01.147408 9945 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:36255 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.201.62:36255: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:01.151683 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.147655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:60493)
0619 12:10:01.147695 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.147704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.147735 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.147882 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.147883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.148032 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.148093 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.148095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.148683 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.148684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.149161 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.149162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.149526 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.149527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.149595 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.149936 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.149941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.151612 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 12:10:01.151751 9684 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.147703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:60493 (local address 127.0.201.62:36255)
0619 12:10:01.147766 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.147768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.147782 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:10:01.147800 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.147800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.147808 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.147864 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.148163 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.148606 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.149256 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.149451 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.149516 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.151677 (+ 2161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:60493: BlockingRecv error: recv got EOF from 127.0.201.3:60493 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:10:01.154564 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.147420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.0.201.62:36255)
0619 12:10:01.147456 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.147458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.147679 (+ 221us) server_negotiation.cc:407] Connection header received
0619 12:10:01.147793 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.147794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.147803 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.147830 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.148551 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.148991 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.149761 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.149999 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.150068 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.151410 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:01.151414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:01.151414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:01.151431 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:01.153201 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:01.153520 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:01.153523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:01.153524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:01.153636 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:01.153817 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:01.153818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:01.153818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:01.154086 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:01.154188 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:01.154371 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 12:10:01.154480 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:01.156364 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.152086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:36985)
0619 12:10:01.152192 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.152203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.152251 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.152504 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.152505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.152663 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.152698 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.152700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.153251 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.153252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.153963 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.153965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.154347 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.154348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.154398 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.154756 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.154760 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.156329 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260619 12:10:01.156469 9684 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.152145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:36985 (local address 127.0.201.62:36255)
0619 12:10:01.152200 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.152202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.152409 (+ 207us) server_negotiation.cc:407] Connection header received
0619 12:10:01.152427 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.152427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.152435 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.152478 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.152775 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.153180 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.154026 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.154262 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.154334 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.156386 (+ 2052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:36985: BlockingRecv error: recv got EOF from 127.0.201.3:36985 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 12:10:01.156481 9945 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:01 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:01.920117 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:01.915514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:59089)
0619 12:10:01.915633 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:01.915643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:01.915701 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:01.915854 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:01.915854 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:01.916002 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:01.916047 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.916049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.916681 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.916682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.917199 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:01.917200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.917526 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.917527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:01.917564 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.917979 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:01.917983 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:01.919981 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
W20260619 12:10:01.920166 9950 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:01.915604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:59089 (local address 127.0.201.62:36255)
0619 12:10:01.915731 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:01.915733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:01.915742 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:01.915765 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:01.915765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:01.915775 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:01.915840 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:01.916113 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.916617 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.917272 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:01.917464 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:01.917517 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:01.920092 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:59089: BlockingRecv error: recv got EOF from 127.0.201.1:59089 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
Jun 19 12:10:02 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:02 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:02.034045 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:02.029324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:40375)
0619 12:10:02.029598 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:02.029610 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:02.029693 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:02.029877 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:02.029877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:02.030050 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:02.030094 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:02.030096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.030759 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.030761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:02.031342 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:02.031344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.031719 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.031720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:02.031783 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:02.032287 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:02.032291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:02.033943 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":188,"threads_started":1}
W20260619 12:10:02.034090 9950 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:02.029507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:40375 (local address 127.0.201.62:36255)
0619 12:10:02.029584 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:02.029586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:02.029676 (+ 90us) server_negotiation.cc:407] Connection header received
0619 12:10:02.029798 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:02.029799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:02.029812 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:02.029858 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:02.030171 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.030664 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.031437 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.031629 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.031710 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:02.034024 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:40375: BlockingRecv error: recv got EOF from 127.0.201.2:40375 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 12:10:02 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:02 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:02.161926 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:02.156827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:45069)
0619 12:10:02.156982 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:02.157003 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:02.157055 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:02.157198 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:02.157198 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:02.157355 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:02.157402 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:02.157404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.158212 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.158213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:02.159044 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:02.159047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.159443 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.159444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:02.159489 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:02.160068 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:02.160073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:02.161861 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
W20260619 12:10:02.162048 9950 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:02.156939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:45069 (local address 127.0.201.62:36255)
0619 12:10:02.157031 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:02.157033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:02.157042 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:02.157132 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:02.157133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:02.157145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:02.157193 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:02.157497 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.158101 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.159121 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.159328 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.159444 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:02.161954 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:45069: BlockingRecv error: recv got EOF from 127.0.201.3:45069 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 12:10:02 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:02 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:02.925408 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:02.920678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:34413)
0619 12:10:02.920817 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:02.920830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:02.920895 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:02.921037 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:02.921037 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:02.921191 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:02.921239 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:02.921240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.921898 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.921899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:02.922413 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:02.922415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.922833 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.922834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:02.922874 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:02.923386 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:02.923393 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:02.925344 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":102,"threads_started":1}
W20260619 12:10:02.925504 9954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:02.920745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:34413 (local address 127.0.201.62:36255)
0619 12:10:02.920883 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:02.920885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:02.920895 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:10:02.920952 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:02.920953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:02.920964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:02.921015 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:02.921303 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.921828 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.922521 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:02.922752 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:02.922822 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:02.925431 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:34413: BlockingRecv error: recv got EOF from 127.0.201.1:34413 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
Jun 19 12:10:03 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:03 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:03.039343 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:03.034573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:42041)
0619 12:10:03.034705 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:03.034736 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:03.034785 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:03.034930 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:03.034931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:03.035082 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:03.035128 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:03.035130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.035868 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.035869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:03.036389 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:03.036390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.036781 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.036782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:03.036819 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:03.037315 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:03.037320 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:03.039243 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
W20260619 12:10:03.039386 9954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:03.034668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:42041 (local address 127.0.201.62:36255)
0619 12:10:03.034765 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:03.034767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:03.034777 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:10:03.034846 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:03.034847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:03.034857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:03.034914 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:03.035229 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.035801 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.036490 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.036712 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.036777 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:03.039314 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:42041: BlockingRecv error: recv got EOF from 127.0.201.2:42041 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 12:10:03 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:03 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:03.166746 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:03.162376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:51135)
0619 12:10:03.162503 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:03.162517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:03.162584 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:03.162761 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:03.162762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:03.162919 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:03.162966 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:03.162967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.163640 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.163640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:03.164157 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:03.164159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.164478 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.164478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:03.164515 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:03.164927 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:03.164931 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:03.166642 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
W20260619 12:10:03.166782 9954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:03.162576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:51135 (local address 127.0.201.62:36255)
0619 12:10:03.162646 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:03.162648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:03.162656 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:10:03.162676 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:03.162677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:03.162689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:03.162738 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:03.163051 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.163567 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.164234 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.164414 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.164466 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:03.166719 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:51135: BlockingRecv error: recv got EOF from 127.0.201.3:51135 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 12:10:03 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:03 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:03.930409 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:03.925913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:54483)
0619 12:10:03.926064 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:03.926082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:03.926128 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:03.926280 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:03.926281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:03.926442 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:03.926489 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:03.926491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.927129 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.927130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:03.927732 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:03.927734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.928076 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.928076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:03.928115 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:03.928529 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:03.928534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:03.930313 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
W20260619 12:10:03.930472 9958 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:03.925992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:54483 (local address 127.0.201.62:36255)
0619 12:10:03.926150 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:03.926152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:03.926160 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:10:03.926181 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:03.926182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:03.926191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:03.926266 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:03.926552 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.927043 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.927808 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:03.928005 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:03.928082 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:03.930400 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:54483: BlockingRecv error: recv got EOF from 127.0.201.1:54483 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
Jun 19 12:10:04 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:04 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:04.044623 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:04.039947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:56085)
0619 12:10:04.040099 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:04.040115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:04.040171 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:04.040304 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:04.040305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:04.040455 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:04.040503 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:04.040505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.041147 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.041148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:04.041649 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:04.041650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.042024 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.042025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:04.042085 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:04.042675 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:04.042680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:04.044517 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
W20260619 12:10:04.044656 9958 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:04.040022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:56085 (local address 127.0.201.62:36255)
0619 12:10:04.040092 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:04.040094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:04.040171 (+ 77us) server_negotiation.cc:407] Connection header received
0619 12:10:04.040216 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:04.040216 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:04.040227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:04.040285 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:04.040590 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.041078 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.041717 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.041930 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.042000 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:04.044596 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:56085: BlockingRecv error: recv got EOF from 127.0.201.2:56085 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 12:10:04 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:04 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:04.172160 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:04.167226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:49725)
0619 12:10:04.167358 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:04.167369 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:04.167425 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:04.167547 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:04.167548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:04.167707 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:04.167911 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:04.167914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.168579 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.168580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:04.169095 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:04.169096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.169450 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.169451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:04.169491 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:04.169967 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:04.169972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:04.172063 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
W20260619 12:10:04.172217 9958 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:04.167309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:49725 (local address 127.0.201.62:36255)
0619 12:10:04.167367 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:04.167369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:04.167425 (+ 56us) server_negotiation.cc:407] Connection header received
0619 12:10:04.167465 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:04.167466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:04.167477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:04.167530 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:04.168004 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.168492 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.169185 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.169372 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.169440 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:04.172153 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:49725: BlockingRecv error: recv got EOF from 127.0.201.3:49725 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 12:10:04 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:04 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:04.935370 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:04.930899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:37853)
0619 12:10:04.931042 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:04.931057 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:04.931112 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:04.931261 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:04.931262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:04.931415 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:04.931461 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:04.931462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.932076 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.932077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:04.932620 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:04.932622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.932951 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.932952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:04.932991 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:04.933493 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:04.933498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:04.935313 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
W20260619 12:10:04.935503 9962 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:04.930951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:37853 (local address 127.0.201.62:36255)
0619 12:10:04.931072 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:04.931074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:04.931099 (+ 25us) server_negotiation.cc:407] Connection header received
0619 12:10:04.931167 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:04.931167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:04.931177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:04.931250 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:04.931519 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.932011 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.932701 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:04.932887 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:04.932937 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:04.935396 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:37853: BlockingRecv error: recv got EOF from 127.0.201.1:37853 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
Jun 19 12:10:05 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:05 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:05.053064 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:05.045147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:57669)
0619 12:10:05.045254 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:05.045272 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:05.045328 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:05.045431 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:05.045432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:05.045627 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:05.045674 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:05.045676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.046299 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.046300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:05.046843 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:05.046844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.047142 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.047143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:05.047180 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:05.051163 (+ 3983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:05.051167 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:05.052970 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
W20260619 12:10:05.053117 9962 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:05.045190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:57669 (local address 127.0.201.62:36255)
0619 12:10:05.045251 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:05.045254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:05.045314 (+ 60us) server_negotiation.cc:407] Connection header received
0619 12:10:05.045379 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:05.045380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:05.045390 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:05.045428 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:05.045739 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.046236 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.046911 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.047083 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.047130 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:05.053051 (+ 5921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:57669: BlockingRecv error: recv got EOF from 127.0.201.2:57669 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 12:10:05 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:05 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:05.178222 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:05.172597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:56575)
0619 12:10:05.172709 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:05.172721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:05.172780 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:05.172928 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:05.172929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:05.173152 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:05.173209 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:05.173211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.174104 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.174106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:05.174972 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:05.174988 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.175540 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.175541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:05.175602 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:05.176142 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:05.176147 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:05.178080 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":55,"threads_started":1}
W20260619 12:10:05.178251 9962 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:05.172632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:56575 (local address 127.0.201.62:36255)
0619 12:10:05.172676 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:05.172678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:05.172781 (+ 103us) server_negotiation.cc:407] Connection header received
0619 12:10:05.172826 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:05.172827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:05.172841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:05.172927 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:05.173309 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.174025 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.175085 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.175438 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.175504 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:05.178183 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:56575: BlockingRecv error: recv got EOF from 127.0.201.3:56575 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 12:10:05 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:05 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:05.940099 9965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:05.935785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:42719)
0619 12:10:05.935896 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:05.935908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:05.935970 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:05.936112 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:05.936113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:05.936264 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:05.936310 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:05.936311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.936957 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.936957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:05.937482 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:05.937483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.937813 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.937814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:05.937852 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:05.938280 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:05.938285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:05.940035 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":47,"threads_started":1}
W20260619 12:10:05.940266 9966 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:05.935842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:42719 (local address 127.0.201.62:36255)
0619 12:10:05.935984 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:05.935986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:05.936012 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:10:05.936036 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:05.936036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:05.936046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:05.936099 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:05.936371 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.936894 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.937567 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:05.937748 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:05.937802 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:05.940157 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:42719: BlockingRecv error: recv got EOF from 127.0.201.1:42719 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
Jun 19 12:10:06 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:06 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:06.058444 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:06.053673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:56437)
0619 12:10:06.053800 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:06.053810 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:06.053855 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:06.054109 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:06.054110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:06.054296 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:06.054341 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:06.054343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.055052 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.055053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:06.055651 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:06.055653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.056045 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.056046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:06.056107 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:06.056566 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:06.056570 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:06.058338 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
W20260619 12:10:06.058476 9966 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:06.053915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:56437 (local address 127.0.201.62:36255)
0619 12:10:06.053974 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:06.053976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:06.053982 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:10:06.054003 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:06.054004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:06.054014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:06.054099 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:06.054430 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.054956 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.055747 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.055957 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.056038 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:06.058413 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:56437: BlockingRecv error: recv got EOF from 127.0.201.2:56437 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 12:10:06 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:06 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:06.183404 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:06.178823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:52941)
0619 12:10:06.178956 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:06.178969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:06.179035 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:06.179180 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:06.179181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:06.179355 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:06.179403 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:06.179404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.180173 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.180175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:06.180714 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:06.180717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.181098 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.181099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:06.181162 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:06.181573 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:06.181578 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:06.183349 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
W20260619 12:10:06.183499 9966 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:06.178922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:52941 (local address 127.0.201.62:36255)
0619 12:10:06.179016 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:06.179018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:06.179028 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:10:06.179095 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:06.179096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:06.179107 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:06.179162 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:06.179467 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.180094 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.180818 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.181022 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.181099 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:06.183436 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:52941: BlockingRecv error: recv got EOF from 127.0.201.3:52941 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 12:10:06 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:06 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:06.945115 9969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:06.940604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:49429)
0619 12:10:06.940757 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:06.940767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:06.940811 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:06.940952 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:06.940952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:06.941109 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:06.941157 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:06.941158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.941985 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.941986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:06.942503 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:06.942504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.942813 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.942814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:06.942852 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:06.943306 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:06.943310 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:06.945021 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
W20260619 12:10:06.945176 9970 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:06.940711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:49429 (local address 127.0.201.62:36255)
0619 12:10:06.940830 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:06.940833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:06.940841 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:10:06.940860 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:06.940861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:06.940871 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:06.940933 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:06.941243 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.941918 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.942572 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:06.942749 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:06.942794 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:06.945096 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:49429: BlockingRecv error: recv got EOF from 127.0.201.1:49429 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
Jun 19 12:10:07 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:07 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:07.063848 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:07.059002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:40735)
0619 12:10:07.059114 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:07.059127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:07.059215 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:07.059434 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:07.059436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:07.059611 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:07.059674 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:07.059676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.060376 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.060377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:07.060989 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:07.060991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.061375 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.061376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:07.061431 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:07.061929 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:07.061933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:07.063754 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
W20260619 12:10:07.063901 9970 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:07.059073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:40735 (local address 127.0.201.62:36255)
0619 12:10:07.059118 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:07.059120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:07.059211 (+ 91us) server_negotiation.cc:407] Connection header received
0619 12:10:07.059281 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:07.059282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:07.059310 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:07.059427 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:07.059754 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.060295 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.061085 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.061292 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.061374 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:07.063837 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:40735: BlockingRecv error: recv got EOF from 127.0.201.2:40735 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 12:10:07 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:07 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:07.188880 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:07.183903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:36699)
0619 12:10:07.184093 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:07.184106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:07.184196 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:07.184324 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:07.184324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:07.184525 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:07.184571 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:07.184573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.185240 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.185242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:07.185773 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:07.185775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.186149 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.186150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:07.186227 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:07.186710 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:07.186715 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:07.188772 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
W20260619 12:10:07.188911 9970 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:07.184003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:36699 (local address 127.0.201.62:36255)
0619 12:10:07.184077 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:07.184079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:07.184178 (+ 99us) server_negotiation.cc:407] Connection header received
0619 12:10:07.184251 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:07.184252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:07.184266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:07.184319 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:07.184652 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.185149 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.185858 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.186062 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.186146 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:07.188847 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:36699: BlockingRecv error: recv got EOF from 127.0.201.3:36699 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 12:10:07 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:07 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:07.949738 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:07.945637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:49397)
0619 12:10:07.945753 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:07.945763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:07.945820 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:07.945956 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:07.945956 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:07.946106 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:07.946152 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:07.946153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.946786 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.946787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:07.947325 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:07.947326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.947635 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.947636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:07.947681 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:07.948090 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:07.948094 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:07.949682 (+ 1588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
W20260619 12:10:07.949852 9974 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:07.945707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:49397 (local address 127.0.201.62:36255)
0619 12:10:07.945841 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:07.945843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:07.945852 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:07.945881 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:07.945882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:07.945891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:07.945952 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:07.946217 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.946718 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.947388 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:07.947574 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:07.947624 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:07.949771 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:49397: BlockingRecv error: recv got EOF from 127.0.201.1:49397 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
Jun 19 12:10:08 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:08 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:08.068760 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:08.064282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:48357)
0619 12:10:08.064453 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:08.064463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:08.064548 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:08.064688 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:08.064688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:08.064857 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:08.064900 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:08.064902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.065573 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.065574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:08.066104 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:08.066105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.066466 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.066467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:08.066527 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:08.066975 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:08.066979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:08.068703 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
W20260619 12:10:08.068881 9974 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:08.064366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:48357 (local address 127.0.201.62:36255)
0619 12:10:08.064437 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:08.064439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:08.064529 (+ 90us) server_negotiation.cc:407] Connection header received
0619 12:10:08.064611 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:08.064612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:08.064623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:08.064680 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:08.064983 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.065486 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.066188 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.066394 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.066469 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:08.068784 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:48357: BlockingRecv error: recv got EOF from 127.0.201.2:48357 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 12:10:08 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:08 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:08.193930 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:08.189380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:58721)
0619 12:10:08.189539 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:08.189552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:08.189624 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:08.189757 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:08.189758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:08.189905 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:08.189953 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:08.189955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.190662 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.190663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:08.191172 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:08.191173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.191562 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.191563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:08.191603 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:08.192045 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:08.192050 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:08.193873 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
W20260619 12:10:08.194061 9974 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:08.189462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:58721 (local address 127.0.201.62:36255)
0619 12:10:08.189550 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:08.189553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:08.189604 (+ 51us) server_negotiation.cc:407] Connection header received
0619 12:10:08.189684 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:08.189685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:08.189696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:08.189743 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:08.190040 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.190582 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.191292 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.191499 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.191555 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:08.193984 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:58721: BlockingRecv error: recv got EOF from 127.0.201.3:58721 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 19 12:10:08 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:08 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:08.954425 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:08.950227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:39261)
0619 12:10:08.950365 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:08.950376 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:08.950433 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:08.950609 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:08.950609 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:08.950771 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:08.950817 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:08.950819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.951467 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.951468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:08.951979 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:08.951980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.952299 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.952299 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:08.952337 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:08.952753 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:08.952757 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:08.954371 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
W20260619 12:10:08.954562 9978 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:08.950343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:39261 (local address 127.0.201.62:36255)
0619 12:10:08.950463 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:08.950465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:08.950505 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:10:08.950525 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:08.950526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:08.950535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:08.950595 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:08.950886 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.951404 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.952049 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:08.952235 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:08.952287 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:08.954478 (+ 2191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:39261: BlockingRecv error: recv got EOF from 127.0.201.1:39261 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
Jun 19 12:10:09 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:09 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:09.074105 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:09.069203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:53357)
0619 12:10:09.069370 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:09.069381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:09.069454 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:09.069643 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:09.069644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:09.069826 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:09.069870 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:09.069872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.070686 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.070687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:09.071292 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:09.071293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.071677 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.071678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:09.071732 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:09.072206 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:09.072230 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:09.074010 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
W20260619 12:10:09.074143 9978 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:09.069267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:53357 (local address 127.0.201.62:36255)
0619 12:10:09.069349 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:09.069351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:09.069447 (+ 96us) server_negotiation.cc:407] Connection header received
0619 12:10:09.069554 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:09.069555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:09.069568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:09.069639 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:09.069956 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.070602 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.071384 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.071582 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.071677 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:09.074084 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:53357: BlockingRecv error: recv got EOF from 127.0.201.2:53357 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 12:10:09 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:09 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:09.199026 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:09.194357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:40489)
0619 12:10:09.194519 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:09.194531 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:09.194634 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:09.194762 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:09.194763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:09.194934 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:09.194983 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:09.194985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.195732 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.195733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:09.196274 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:09.196276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.196658 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.196659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:09.196723 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:09.197244 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:09.197253 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:09.198958 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
W20260619 12:10:09.199095 9978 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:09.194427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:40489 (local address 127.0.201.62:36255)
0619 12:10:09.194500 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:09.194502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:09.194618 (+ 116us) server_negotiation.cc:407] Connection header received
0619 12:10:09.194690 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:09.194691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:09.194705 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:09.194757 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:09.195081 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.195650 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.196361 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.196573 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.196658 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:09.199048 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:40489: BlockingRecv error: recv got EOF from 127.0.201.3:40489 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 12:10:09 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:09 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:09.959559 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:09.954969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:60079)
0619 12:10:09.955121 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:09.955132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:09.955185 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:09.955374 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:09.955375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:09.955534 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:09.955580 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:09.955582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.956194 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.956195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:09.956722 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:09.956724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.957042 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.957042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:09.957083 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:09.957600 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:09.957605 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:09.959503 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
W20260619 12:10:09.959708 9982 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:09.955059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:60079 (local address 127.0.201.62:36255)
0619 12:10:09.955177 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:09.955179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:09.955189 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:10:09.955292 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:09.955293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:09.955303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:09.955360 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:09.955653 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.956131 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.956792 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:09.956971 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:09.957018 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:09.959581 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:60079: BlockingRecv error: recv got EOF from 127.0.201.1:60079 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":87,"threads_started":1}
Jun 19 12:10:10 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:10 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:10.084427 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:10.074611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:51241)
0619 12:10:10.074824 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:10.074836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:10.074928 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:10.075110 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:10.075111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:10.075349 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:10.075394 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:10.075396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.076147 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.076148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:10.076683 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:10.076685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.077067 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.077068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:10.077126 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:10.082327 (+ 5201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:10.082333 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:10.084324 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
W20260619 12:10:10.084502 9982 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:10.074725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:51241 (local address 127.0.201.62:36255)
0619 12:10:10.074810 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:10.074813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:10.074908 (+ 95us) server_negotiation.cc:407] Connection header received
0619 12:10:10.075016 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:10.075017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:10.075030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:10.075103 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:10.075530 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.076060 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.076782 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.076980 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.077067 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:10.084432 (+ 7365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:51241: BlockingRecv error: recv got EOF from 127.0.201.2:51241 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 12:10:10 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:10 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:10.204274 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:10.199665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:42643)
0619 12:10:10.199810 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:10.199820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:10.199880 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:10.199996 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:10.199997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:10.200161 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:10.200206 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:10.200207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.200832 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.200833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:10.201343 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:10.201344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.201674 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.201675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:10.201714 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:10.202165 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:10.202170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:10.204180 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260619 12:10:10.204375 9982 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:10.199719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:42643 (local address 127.0.201.62:36255)
0619 12:10:10.199784 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:10.199786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:10.199879 (+ 93us) server_negotiation.cc:407] Connection header received
0619 12:10:10.199921 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:10.199922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:10.199933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:10.199981 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:10.200263 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.200770 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.201411 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.201616 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.201668 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:10.204269 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:42643: BlockingRecv error: recv got EOF from 127.0.201.3:42643 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 12:10:10 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:10 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:10.964641 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:10.960051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:59419)
0619 12:10:10.960172 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:10.960191 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:10.960239 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:10.960388 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:10.960389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:10.960568 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:10.960615 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:10.960617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.961320 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.961321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:10.961856 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:10.961858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.962207 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.962208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:10.962258 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:10.962787 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:10.962793 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:10.964576 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
W20260619 12:10:10.964745 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:10.960128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:59419 (local address 127.0.201.62:36255)
0619 12:10:10.960257 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:10.960259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:10.960268 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:10.960288 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:10.960288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:10.960298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:10.960360 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:10.960694 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.961227 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.961925 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:10.962111 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:10.962164 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:10.964669 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:59419: BlockingRecv error: recv got EOF from 127.0.201.1:59419 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
Jun 19 12:10:11 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:11 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:11.089954 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:11.085061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:51991)
0619 12:10:11.085225 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:11.085238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:11.085300 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:11.085484 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:11.085485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:11.085675 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:11.085722 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:11.085724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.086426 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.086428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:11.086971 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:11.086973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.087428 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.087429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:11.087489 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:11.087977 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:11.087985 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:11.089844 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
W20260619 12:10:11.090004 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:11.085096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:51991 (local address 127.0.201.62:36255)
0619 12:10:11.085144 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:11.085146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:11.085284 (+ 138us) server_negotiation.cc:407] Connection header received
0619 12:10:11.085358 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:11.085359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:11.085372 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:11.085475 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:11.085806 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.086334 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.087073 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.087340 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.087431 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:11.089929 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:51991: BlockingRecv error: recv got EOF from 127.0.201.2:51991 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 12:10:11 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:11 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:11.208932 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:11.204672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:44255)
0619 12:10:11.204823 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:11.204841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:11.204896 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:11.205013 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:11.205013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:11.205184 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:11.205230 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:11.205231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.205839 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.205840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:11.206356 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:11.206357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.206693 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.206694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:11.206731 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:11.207164 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:11.207168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:11.208868 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":56,"threads_started":1}
W20260619 12:10:11.209049 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:11.204781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:44255 (local address 127.0.201.62:36255)
0619 12:10:11.204856 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:11.204858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:11.204881 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:10:11.204938 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:11.204939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:11.204950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:11.204998 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:11.205290 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.205778 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.206431 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.206621 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.206684 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:11.208952 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:44255: BlockingRecv error: recv got EOF from 127.0.201.3:44255 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 12:10:11 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:11 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:11.969623 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:11.965085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:60529)
0619 12:10:11.965229 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:11.965238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:11.965286 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:11.965432 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:11.965432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:11.965706 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:11.965766 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:11.965769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.966491 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.966492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:11.967008 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:11.967009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.967362 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.967363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:11.967405 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:11.967817 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:11.967822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:11.969548 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
W20260619 12:10:11.969771 9990 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:11.965181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:60529 (local address 127.0.201.62:36255)
0619 12:10:11.965303 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:11.965305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:11.965314 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:11.965334 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:11.965335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:11.965344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:11.965406 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:11.965856 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.966418 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.967086 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:11.967298 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:11.967351 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:11.969649 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:60529: BlockingRecv error: recv got EOF from 127.0.201.1:60529 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
Jun 19 12:10:12 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:12 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:12.095407 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:12.090454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:54715)
0619 12:10:12.090637 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:12.090649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:12.090738 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:12.090907 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:12.090908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:12.091094 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:12.091140 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:12.091141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.091834 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.091835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:12.092354 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:12.092356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.092733 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.092734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:12.092784 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:12.093242 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:12.093247 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:12.095297 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
W20260619 12:10:12.095435 9990 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:12.090566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54715 (local address 127.0.201.62:36255)
0619 12:10:12.090637 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:12.090639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:12.090720 (+ 81us) server_negotiation.cc:407] Connection header received
0619 12:10:12.090821 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:12.090822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:12.090835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:12.090902 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:12.091242 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.091751 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.092435 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.092634 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.092744 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:12.095371 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:54715: BlockingRecv error: recv got EOF from 127.0.201.2:54715 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 12:10:12 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:12 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:12.214443 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:12.209392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:49745)
0619 12:10:12.209523 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:12.209536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:12.209611 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:12.209762 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:12.209762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:12.209948 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:12.209996 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:12.209998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.210987 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.210988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:12.211569 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:12.211571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.211896 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.211897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:12.211936 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:12.212487 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:12.212493 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:12.214365 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":98,"threads_started":1}
W20260619 12:10:12.214529 9990 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:12.209463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:49745 (local address 127.0.201.62:36255)
0619 12:10:12.209532 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:12.209535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:12.209627 (+ 92us) server_negotiation.cc:407] Connection header received
0619 12:10:12.209679 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:12.209680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:12.209693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:12.209757 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:12.210173 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.210911 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.211650 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.211826 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.211880 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:12.214444 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:49745: BlockingRecv error: recv got EOF from 127.0.201.3:49745 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 12:10:12 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:12 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:12.974730 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:12.970079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:37831)
0619 12:10:12.970190 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:12.970201 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:12.970258 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:12.970461 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:12.970461 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:12.970634 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:12.970680 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:12.970681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.971381 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.971381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:12.971903 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:12.971904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.972265 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.972266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:12.972305 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:12.972741 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:12.972745 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:12.974625 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
W20260619 12:10:12.974843 9994 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:12.970161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:37831 (local address 127.0.201.62:36255)
0619 12:10:12.970327 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:12.970329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:12.970339 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:10:12.970362 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:12.970363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:12.970373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:12.970451 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:12.970754 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.971316 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.971982 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:12.972198 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:12.972259 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:12.974746 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:37831: BlockingRecv error: recv got EOF from 127.0.201.1:37831 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
Jun 19 12:10:13 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:13 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:13.100616 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:13.095876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:34173)
0619 12:10:13.096080 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:13.096093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:13.096170 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:13.096307 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:13.096307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:13.096493 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:13.096535 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:13.096537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.097236 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.097237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:13.097782 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:13.097784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.098161 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.098162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:13.098215 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:13.098708 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:13.098712 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:13.100539 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":141,"threads_started":1}
W20260619 12:10:13.100679 9994 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:13.095932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:34173 (local address 127.0.201.62:36255)
0619 12:10:13.095977 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:13.095979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:13.096152 (+ 173us) server_negotiation.cc:407] Connection header received
0619 12:10:13.096235 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:13.096236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:13.096250 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:13.096302 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:13.096635 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.097153 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.097870 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.098070 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.098161 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:13.100633 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:34173: BlockingRecv error: recv got EOF from 127.0.201.2:34173 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 12:10:13 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:13 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:13.219837 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:13.214915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:33099)
0619 12:10:13.215086 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:13.215101 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:13.215220 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:13.215387 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:13.215388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:13.215587 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:13.215632 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:13.215634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.216307 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.216308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:13.216850 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:13.216852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.217247 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.217248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:13.217304 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:13.217759 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:13.217763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:13.219738 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":54,"threads_started":1}
W20260619 12:10:13.219879 9994 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:13.215056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:33099 (local address 127.0.201.62:36255)
0619 12:10:13.215115 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:13.215117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:13.215178 (+ 61us) server_negotiation.cc:407] Connection header received
0619 12:10:13.215299 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:13.215300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:13.215317 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:13.215381 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:13.215716 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.216219 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.216960 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.217148 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.217249 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:13.219819 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:33099: BlockingRecv error: recv got EOF from 127.0.201.3:33099 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 12:10:13 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:13 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:13.979811 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:13.975291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:36561)
0619 12:10:13.975427 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:13.975439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:13.975514 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:13.975653 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:13.975654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:13.975833 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:13.975878 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:13.975880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.976502 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.976503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:13.977031 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:13.977033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.977347 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.977348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:13.977393 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:13.977830 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:13.977836 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:13.979704 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
W20260619 12:10:13.979851 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:13.975302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:36561 (local address 127.0.201.62:36255)
0619 12:10:13.975449 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:13.975451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:13.975513 (+ 62us) server_negotiation.cc:407] Connection header received
0619 12:10:13.975570 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:13.975571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:13.975581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:13.975630 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:13.975941 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.976435 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.977108 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:13.977285 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:13.977332 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:13.979779 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:36561: BlockingRecv error: recv got EOF from 127.0.201.1:36561 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
Jun 19 12:10:14 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:14 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:14.106151 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:14.101147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:59053)
0619 12:10:14.101305 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:14.101315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:14.101368 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:14.101555 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:14.101556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:14.101717 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:14.101763 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:14.101765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.102426 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.102428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:14.102985 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:14.102986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.103532 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.103533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:14.103597 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:14.104042 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:14.104047 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:14.106040 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
W20260619 12:10:14.106178 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:14.101287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:59053 (local address 127.0.201.62:36255)
0619 12:10:14.101358 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:14.101360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:14.101368 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:10:14.101420 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:14.101420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:14.101430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:14.101489 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:14.101830 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.102333 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.103105 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.103423 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.103496 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:14.106117 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:59053: BlockingRecv error: recv got EOF from 127.0.201.2:59053 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 12:10:14 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:14 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:14.225342 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:14.220400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:56141)
0619 12:10:14.220555 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:14.220583 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:14.220666 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:14.220897 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:14.220899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:14.221086 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:14.221136 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:14.221138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.221923 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.221925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:14.222521 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:14.222524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.222909 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.222910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:14.222972 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:14.223546 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:14.223551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:14.225274 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
W20260619 12:10:14.225404 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:14.220503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:56141 (local address 127.0.201.62:36255)
0619 12:10:14.220558 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:14.220560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:14.220646 (+ 86us) server_negotiation.cc:407] Connection header received
0619 12:10:14.220768 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:14.220770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:14.220786 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:14.220841 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:14.221210 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.221834 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.222628 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.222824 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.222907 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:14.225355 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:56141: BlockingRecv error: recv got EOF from 127.0.201.3:56141 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 12:10:14 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:14 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:14.984723 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:14.980309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:44975)
0619 12:10:14.980486 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:14.980498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:14.980551 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:14.980668 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:14.980668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:14.980827 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:14.980879 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:14.980881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.981511 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.981513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:14.982067 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:14.982069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.982419 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.982419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:14.982460 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:14.982903 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:14.982908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:14.984653 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
W20260619 12:10:14.984783 10002 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:14.980411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:44975 (local address 127.0.201.62:36255)
0619 12:10:14.980531 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:14.980533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:14.980542 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:14.980597 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:14.980597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:14.980607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:14.980658 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:14.980940 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.981426 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.982142 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:14.982325 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:14.982373 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:14.984731 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:44975: BlockingRecv error: recv got EOF from 127.0.201.1:44975 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
Jun 19 12:10:15 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:15 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:15.116446 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:15.106605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:53429)
0619 12:10:15.106726 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:15.106738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:15.106797 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:15.106921 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:15.106922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:15.107112 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:15.107156 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:15.107157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.107869 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.107870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:15.108427 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:15.108428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.108795 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.108796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:15.108851 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:15.114403 (+ 5552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:15.114408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:15.116345 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
W20260619 12:10:15.116519 10002 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:15.106727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:53429 (local address 127.0.201.62:36255)
0619 12:10:15.106783 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:15.106786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:15.106795 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:15.106854 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:15.106855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:15.106867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:15.106919 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:15.107274 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.107782 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.108521 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.108724 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.108798 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:15.116453 (+ 7655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:53429: BlockingRecv error: recv got EOF from 127.0.201.2:53429 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 19 12:10:15 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:15 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.3@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:15.231384 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:15.225819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.3:46819)
0619 12:10:15.225924 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:15.225937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:15.226002 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:15.226140 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:15.226141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:15.226339 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:15.226389 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:15.226392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.227053 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.227054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:15.227953 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:15.227955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.228322 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.228323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:15.228382 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:15.228896 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:15.228901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:15.231275 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":57,"threads_started":1}
W20260619 12:10:15.231441 10002 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:15.225848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:46819 (local address 127.0.201.62:36255)
0619 12:10:15.225937 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:15.225939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:15.225982 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:10:15.226070 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:15.226070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:15.226085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:15.226134 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:15.226472 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.226971 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.228033 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.228245 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.228322 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:15.231362 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.3:46819: BlockingRecv error: recv got EOF from 127.0.201.3:46819 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 12:10:15 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:15 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.1@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:15.989889 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:15.985295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.1:49063)
0619 12:10:15.985420 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:15.985434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:15.985500 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:15.985623 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:15.985624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:15.985783 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:15.985832 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:15.985833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.986540 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.986541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:15.987060 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:15.987062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.987459 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.987460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:15.987500 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:15.988007 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:15.988011 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:15.989822 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
W20260619 12:10:15.990079 10006 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:15.985366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:49063 (local address 127.0.201.62:36255)
0619 12:10:15.985489 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:15.985491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:15.985501 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:10:15.985548 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:15.985548 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:15.985560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:15.985620 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:15.985944 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.986467 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.987130 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:15.987395 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:15.987452 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:15.989990 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.1:49063: BlockingRecv error: recv got EOF from 127.0.201.1:49063 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
Jun 19 12:10:16 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:10:16 dist-test-slave-c910 krb5kdc[8948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.201.2@KRBTEST.COM for oryx/127.0.201.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:10:16.121827 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.116999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36255 (local address 127.0.201.2:48249)
0619 12:10:16.117188 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:16.117199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:16.117283 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:16.117554 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:16.117555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:16.117784 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:16.117860 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.117862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.118517 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.118518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.119031 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.119033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.119426 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.119427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.119468 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.119943 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:16.119947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:16.121724 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.201.62:36255: Server oryx/127.0.201.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
W20260619 12:10:16.121861 10006 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:16.117091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:48249 (local address 127.0.201.62:36255)
0619 12:10:16.117156 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.117158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.117348 (+ 190us) server_negotiation.cc:407] Connection header received
0619 12:10:16.117382 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.117383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.117396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.117457 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.117946 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.118453 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.119101 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.119344 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.119396 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.121798 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.201.2:48249: BlockingRecv error: recv got EOF from 127.0.201.2:48249 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:10:16.149746 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9531
I20260619 12:10:16.154124 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9683
I20260619 12:10:16.158468 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9816
I20260619 12:10:16.162703 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 9461
2026-06-19T12:10:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17460 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T12:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:10:16Z Disabled control of system clock
I20260619 12:10:16.189365 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:41307
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:41307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 12:10:16.268740 10011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:16.268949 10011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:16.268970 10011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:16.270192 10011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:10:16.270238 10011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:16.270252 10011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:16.270264 10011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:10:16.270277 10011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:10:16.271692 10011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:41307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:41307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:16.271944 10011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:16.272260 10011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:10:16.277385 10011 server_base.cc:1034] running on GCE node
W20260619 12:10:16.277367 10017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.277352 10016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.277346 10019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:16.277827 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:16.278156 10011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:16.279261 10011 hybrid_clock.cc:648] HybridClock initialized: now 1781871016279247 us; error 28 us; skew 500 ppm
I20260619 12:10:16.280568 10011 webserver.cc:460] Webserver started at http://127.0.201.62:41241/ using document root <none> and password file <none>
I20260619 12:10:16.280820 10011 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:16.280869 10011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:16.281021 10011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:16.282087 10011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "32ea6f169cdd47228586c01d2bdf6233"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "2dd31edb8d5be9188def04d2edcd04c8"
server_key_iv: "971c919886c2dce84beaf5c20b9e9729"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.282433 10011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "32ea6f169cdd47228586c01d2bdf6233"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "2dd31edb8d5be9188def04d2edcd04c8"
server_key_iv: "971c919886c2dce84beaf5c20b9e9729"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.283833 10011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:10:16.284575 10025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.284754 10011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:16.284853 10011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "32ea6f169cdd47228586c01d2bdf6233"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "2dd31edb8d5be9188def04d2edcd04c8"
server_key_iv: "971c919886c2dce84beaf5c20b9e9729"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.284926 10011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:16.297320 10011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:16.297617 10011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:16.297765 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:16.301587 10011 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:41307
I20260619 12:10:16.301618 10077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:41307 every 8 connection(s)
I20260619 12:10:16.301962 10011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:10:16.302580 10078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:10:16.303303 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10011
I20260619 12:10:16.303403 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:10:16.303591 804 external_mini_cluster.cc:1428] Setting key 07f934f1a771c332a7c52ef8c7e72ee2
I20260619 12:10:16.305590 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233: Bootstrap starting.
I20260619 12:10:16.306226 10078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233: Neither blocks nor log segments found. Creating new log.
I20260619 12:10:16.306536 10078 log.cc:824] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:16.307269 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233: No bootstrap required, opened a new log
I20260619 12:10:16.308992 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.304101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.0.201.62:41307)
0619 12:10:16.304272 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.304279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.304303 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:10:16.304706 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.304712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.304769 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.304877 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.305119 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.305988 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.307030 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.307509 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.307607 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.308253 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:16.308256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:16.308256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:16.308413 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:16.308443 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:16.308453 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:16.308502 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:16.308604 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:10:16.308684 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260619 12:10:16.309049 10078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ea6f169cdd47228586c01d2bdf6233" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41307 } }
I20260619 12:10:16.309229 10078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:10:16.309264 10078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ea6f169cdd47228586c01d2bdf6233, State: Initialized, Role: FOLLOWER
I20260619 12:10:16.309386 10078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ea6f169cdd47228586c01d2bdf6233" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41307 } }
I20260619 12:10:16.309466 10078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:10:16.309505 10078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:10:16.309557 10078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:10:16.310077 10078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ea6f169cdd47228586c01d2bdf6233" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41307 } }
I20260619 12:10:16.310189 10078 leader_election.cc:302] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32ea6f169cdd47228586c01d2bdf6233; no voters:
I20260619 12:10:16.310344 10078 leader_election.cc:288] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:10:16.310411 10083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:10:16.310565 10083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [term 1 LEADER]: Becoming Leader. State: Replica: 32ea6f169cdd47228586c01d2bdf6233, State: Running, Role: LEADER
I20260619 12:10:16.310645 10078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:10:16.310699 10083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ea6f169cdd47228586c01d2bdf6233" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41307 } }
I20260619 12:10:16.311086 10085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ea6f169cdd47228586c01d2bdf6233. Latest consensus state: current_term: 1 leader_uuid: "32ea6f169cdd47228586c01d2bdf6233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ea6f169cdd47228586c01d2bdf6233" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41307 } } }
I20260619 12:10:16.311172 10085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:16.311363 10084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32ea6f169cdd47228586c01d2bdf6233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ea6f169cdd47228586c01d2bdf6233" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41307 } } }
I20260619 12:10:16.311441 10091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:10:16.311486 10084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:16.311956 10091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:10:16.313375 10091 catalog_manager.cc:1347] Generated new cluster ID: b28ff32c830e46cf9cc1d673f3178b1f
I20260619 12:10:16.313441 10091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:10:16.347101 10091 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:10:16.347666 10091 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:10:16.359211 10091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32ea6f169cdd47228586c01d2bdf6233: Generated new TSK 0
I20260619 12:10:16.359436 10091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:10:16.361889 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:41307
--builtin_ntp_servers=127.0.201.20:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 12:10:16.454670 10102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:16.454864 10102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:16.454883 10102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:16.456009 10102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:16.456056 10102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:16.456081 10102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:10:16.457366 10102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:41307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:16.457535 10102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:16.457758 10102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:10:16.462597 10102 server_base.cc:1034] running on GCE node
W20260619 12:10:16.462512 10110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.462541 10108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.462520 10107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:16.462921 10102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:16.463116 10102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:16.464254 10102 hybrid_clock.cc:648] HybridClock initialized: now 1781871016464246 us; error 52 us; skew 500 ppm
I20260619 12:10:16.465431 10102 webserver.cc:460] Webserver started at http://127.0.201.1:38769/ using document root <none> and password file <none>
I20260619 12:10:16.465657 10102 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:16.465701 10102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:16.465858 10102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:16.466917 10102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "dd7b7394fd53430d91e5fef29ce7a40e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "4c4bef7ee5869243fc468afb8f402cb9"
server_key_iv: "b521c8dc02aee54e9c93fd45b15d2234"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.467322 10102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "dd7b7394fd53430d91e5fef29ce7a40e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "4c4bef7ee5869243fc468afb8f402cb9"
server_key_iv: "b521c8dc02aee54e9c93fd45b15d2234"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.468612 10102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:16.469295 10116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.469450 10102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:16.469542 10102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "dd7b7394fd53430d91e5fef29ce7a40e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "4c4bef7ee5869243fc468afb8f402cb9"
server_key_iv: "b521c8dc02aee54e9c93fd45b15d2234"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.469624 10102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:16.489841 10102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:16.490131 10102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:16.490285 10102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:16.490502 10102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:16.490805 10102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:16.490857 10102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.490896 10102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:16.490924 10102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.496591 10102 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:40819
I20260619 12:10:16.496629 10229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:40819 every 8 connection(s)
I20260619 12:10:16.496941 10102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:10:16.497557 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10102
I20260619 12:10:16.497653 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:10:16.497876 804 external_mini_cluster.cc:1428] Setting key 6661c554cfacb869d66ca0d1a56a0693
I20260619 12:10:16.499167 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:41307
--builtin_ntp_servers=127.0.201.20:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 12:10:16.503589 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.497947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:51079 (local address 127.0.201.62:41307)
0619 12:10:16.497997 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.497999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.498239 (+ 240us) server_negotiation.cc:407] Connection header received
0619 12:10:16.498901 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.498902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.498915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.498996 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.500298 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.501077 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.502012 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.502355 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.502436 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.503284 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:16.503286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:16.503286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:16.503291 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:16.503307 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:16.503312 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:16.503383 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:16.503436 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:10:16.503468 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:10:16.503690 10232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.497650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41307 (local address 127.0.201.1:51079)
0619 12:10:16.498050 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:16.498092 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:16.498780 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:16.499165 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:16.499170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:16.499181 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:16.500111 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.500118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.501168 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.501169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.501924 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.501926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.502442 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.502443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.502546 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.503121 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:10:16.503122 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:16.503149 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:10:16.503170 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:16.503382 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:16.503385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:16.503388 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:10:16.503434 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:10:16.503507 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":86,"threads_started":1}
I20260619 12:10:16.504208 10230 heartbeater.cc:347] Connected to a master server at 127.0.201.62:41307
I20260619 12:10:16.504314 10230 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:16.504478 10230 heartbeater.cc:511] Master 127.0.201.62:41307 requested a full tablet report, sending...
I20260619 12:10:16.504978 10042 ts_manager.cc:194] Registered new tserver with Master: dd7b7394fd53430d91e5fef29ce7a40e (127.0.201.1:40819)
I20260619 12:10:16.505580 10042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.1:51079
W20260619 12:10:16.580719 10233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:16.580912 10233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:16.580933 10233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:16.582135 10233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:16.582175 10233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:16.582221 10233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:10:16.583628 10233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:41307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:16.583891 10233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:16.584183 10233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:16.588989 10241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.589030 10239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.589051 10238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:16.589174 10233 server_base.cc:1034] running on GCE node
I20260619 12:10:16.589414 10233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:16.589603 10233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:16.590735 10233 hybrid_clock.cc:648] HybridClock initialized: now 1781871016590723 us; error 29 us; skew 500 ppm
I20260619 12:10:16.591933 10233 webserver.cc:460] Webserver started at http://127.0.201.2:38415/ using document root <none> and password file <none>
I20260619 12:10:16.592162 10233 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:16.592208 10233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:16.592348 10233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:16.593366 10233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "9c5567fd74294c33b9609d6a56c0180e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "b892d397dfe21b135b1463f5e9d17a40"
server_key_iv: "3f0ce72c1b23c129653b0d82f6f74d04"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.593714 10233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "9c5567fd74294c33b9609d6a56c0180e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "b892d397dfe21b135b1463f5e9d17a40"
server_key_iv: "3f0ce72c1b23c129653b0d82f6f74d04"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.594988 10233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:10:16.595757 10247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.595917 10233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.596010 10233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "9c5567fd74294c33b9609d6a56c0180e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "b892d397dfe21b135b1463f5e9d17a40"
server_key_iv: "3f0ce72c1b23c129653b0d82f6f74d04"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.596100 10233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:16.603291 10233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:16.603546 10233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:16.603689 10233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:16.603914 10233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:16.604208 10233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:16.604262 10233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.604301 10233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:16.604328 10233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.610049 10233 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:41155
I20260619 12:10:16.610080 10360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:41155 every 8 connection(s)
I20260619 12:10:16.610380 10233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:10:16.614183 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10233
I20260619 12:10:16.614275 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:10:16.614462 804 external_mini_cluster.cc:1428] Setting key 92b8f9bdf5c83139713e49dfc3fb506a
I20260619 12:10:16.615557 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:41307
--builtin_ntp_servers=127.0.201.20:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 12:10:16.615983 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.611119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:50185 (local address 127.0.201.62:41307)
0619 12:10:16.611186 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.611188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.611420 (+ 232us) server_negotiation.cc:407] Connection header received
0619 12:10:16.611952 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.611954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.611964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.612031 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.612684 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.613169 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.614099 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.614460 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.614545 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.615575 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:16.615576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:16.615576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:16.615580 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:16.615600 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:16.615604 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:16.615684 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:16.615873 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 12:10:16.615904 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:10:16.616199 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.611102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41307 (local address 127.0.201.2:50185)
0619 12:10:16.611289 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:16.611346 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:16.611860 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:16.612084 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:16.612087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:16.612094 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:16.612609 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.612614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.613240 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.613241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.614002 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.614004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.614568 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.614569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.614723 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.615417 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:10:16.615417 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:16.615453 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:10:16.615481 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:16.615766 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:16.615773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:16.615776 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:10:16.615875 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:10:16.615994 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":144,"threads_started":1}
I20260619 12:10:16.617134 10361 heartbeater.cc:347] Connected to a master server at 127.0.201.62:41307
I20260619 12:10:16.617220 10361 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:16.617404 10361 heartbeater.cc:511] Master 127.0.201.62:41307 requested a full tablet report, sending...
I20260619 12:10:16.617766 10042 ts_manager.cc:194] Registered new tserver with Master: 9c5567fd74294c33b9609d6a56c0180e (127.0.201.2:41155)
I20260619 12:10:16.618132 10042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.2:50185
W20260619 12:10:16.694734 10364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:16.694957 10364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:16.694990 10364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:16.696112 10364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:16.696175 10364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:16.696215 10364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:10:16.697502 10364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:41307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:16.697775 10364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:16.698066 10364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:10:16.703047 10364 server_base.cc:1034] running on GCE node
W20260619 12:10:16.703068 10370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.703063 10369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.703039 10372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:16.703523 10364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:16.703724 10364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:16.704870 10364 hybrid_clock.cc:648] HybridClock initialized: now 1781871016704862 us; error 32 us; skew 500 ppm
I20260619 12:10:16.706110 10364 webserver.cc:460] Webserver started at http://127.0.201.3:42203/ using document root <none> and password file <none>
I20260619 12:10:16.706350 10364 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:16.706395 10364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:16.706518 10364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:16.707620 10364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "692a132247b64054937718886ab3059e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "67a5144358f82b64702682d5fde25092"
server_key_iv: "331ea07f83b1c59794311dc2cbff2380"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.707962 10364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "692a132247b64054937718886ab3059e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "67a5144358f82b64702682d5fde25092"
server_key_iv: "331ea07f83b1c59794311dc2cbff2380"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.709221 10364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:10:16.709920 10378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.710146 10364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:16.710215 10364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "692a132247b64054937718886ab3059e"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "67a5144358f82b64702682d5fde25092"
server_key_iv: "331ea07f83b1c59794311dc2cbff2380"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.710305 10364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:16.720978 10364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:16.721278 10364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:16.721426 10364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:16.721652 10364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:16.721980 10364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:16.722033 10364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.722069 10364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:16.722096 10364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.727753 10364 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:38005
I20260619 12:10:16.727792 10491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:38005 every 8 connection(s)
I20260619 12:10:16.728099 10364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:10:16.730443 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10364
I20260619 12:10:16.730554 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:10:16.730746 804 external_mini_cluster.cc:1428] Setting key 4d8f3e6972d2014e5a0ca8ffd7c87ab8
I20260619 12:10:16.733415 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.728796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:47147 (local address 127.0.201.62:41307)
0619 12:10:16.728853 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.728855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.729098 (+ 243us) server_negotiation.cc:407] Connection header received
0619 12:10:16.729632 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.729633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.729646 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.729692 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.730362 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.730980 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.732041 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.732247 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.732333 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.733140 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:16.733142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:16.733142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:16.733147 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:16.733161 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:16.733164 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:16.733218 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:16.733313 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 12:10:16.733337 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:10:16.733616 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.728790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41307 (local address 127.0.201.3:47147)
0619 12:10:16.728982 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:16.729016 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:16.729513 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:16.729772 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:16.729775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:16.729783 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:16.730272 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.730276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.731087 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.731088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.731938 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:16.731940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:16.732341 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:16.732342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:16.732471 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:16.733006 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:10:16.733006 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:16.733033 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:10:16.733056 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:16.733221 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:16.733225 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:16.733228 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:10:16.733314 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:10:16.733409 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260619 12:10:16.734097 10492 heartbeater.cc:347] Connected to a master server at 127.0.201.62:41307
I20260619 12:10:16.734170 10492 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:16.734372 10492 heartbeater.cc:511] Master 127.0.201.62:41307 requested a full tablet report, sending...
I20260619 12:10:16.734737 10042 ts_manager.cc:194] Registered new tserver with Master: 692a132247b64054937718886ab3059e (127.0.201.3:38005)
I20260619 12:10:16.735090 10042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.3:47147
I20260619 12:10:16.742543 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:10:16.744469 10081 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:16.743974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.0.201.62:41307)
0619 12:10:16.744041 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.744043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.744190 (+ 147us) server_negotiation.cc:407] Connection header received
0619 12:10:16.744236 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.744236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.744244 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.744281 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.744416 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46142: BlockingRecv error: recv got EOF from 127.0.0.1:46142 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:10:16.744547 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10102
I20260619 12:10:16.748739 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10233
I20260619 12:10:16.752851 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10364
I20260619 12:10:16.756799 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10011
2026-06-19T12:10:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (593 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T12:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:10:16Z Disabled control of system clock
I20260619 12:10:16.783555 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45495
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:10:16.862495 10503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:16.862689 10503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:16.862708 10503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:16.863792 10503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:10:16.863828 10503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:16.863842 10503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:16.863853 10503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:10:16.863866 10503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:10:16.865104 10503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:45495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:16.865295 10503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:16.865531 10503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:10:16.870210 10503 server_base.cc:1034] running on GCE node
W20260619 12:10:16.870194 10511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.870194 10509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:16.870182 10508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:16.870653 10503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:16.870882 10503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:16.872017 10503 hybrid_clock.cc:648] HybridClock initialized: now 1781871016871994 us; error 48 us; skew 500 ppm
I20260619 12:10:16.873075 10503 webserver.cc:460] Webserver started at http://127.0.201.62:46747/ using document root <none> and password file <none>
I20260619 12:10:16.873262 10503 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:16.873306 10503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:16.873390 10503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:16.874400 10503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "bd5107c154374ca5909bd274a7d989c3"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "c6e8bf7b2f2813d8b36c60501410bc0d"
server_key_iv: "7455036ffd8063f930356805973b1ecd"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.874691 10503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "bd5107c154374ca5909bd274a7d989c3"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "c6e8bf7b2f2813d8b36c60501410bc0d"
server_key_iv: "7455036ffd8063f930356805973b1ecd"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.876021 10503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:10:16.876667 10517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:16.876822 10503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:16.876916 10503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "bd5107c154374ca5909bd274a7d989c3"
format_stamp: "Formatted at 2026-06-19 12:10:16 on dist-test-slave-c910"
server_key: "c6e8bf7b2f2813d8b36c60501410bc0d"
server_key_iv: "7455036ffd8063f930356805973b1ecd"
server_key_version: "encryptionkey@0"
I20260619 12:10:16.876996 10503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:16.880281 10503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:16.880534 10503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:16.880679 10503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:16.884385 10503 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:45495
I20260619 12:10:16.884415 10569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:45495 every 8 connection(s)
I20260619 12:10:16.884734 10503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:10:16.885334 10570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:10:16.887085 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10503
I20260619 12:10:16.887184 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:10:16.887413 804 external_mini_cluster.cc:1428] Setting key ecc29551050239f299464a7a3e3a9627
I20260619 12:10:16.888425 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3: Bootstrap starting.
I20260619 12:10:16.889140 10570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3: Neither blocks nor log segments found. Creating new log.
I20260619 12:10:16.889470 10570 log.cc:824] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:16.890070 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:16.887921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.0.201.62:45495)
0619 12:10:16.888128 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:16.888138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:16.888170 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:10:16.888694 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:16.888703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:16.888763 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:16.888927 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:16.889601 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:16.889605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:16.889605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:16.889610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:16.889655 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:16.889666 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:16.889748 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:16.889817 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:10:16.889872 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260619 12:10:16.890235 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3: No bootstrap required, opened a new log
I20260619 12:10:16.891700 10570 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5107c154374ca5909bd274a7d989c3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45495 } }
I20260619 12:10:16.891826 10570 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:10:16.891848 10570 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5107c154374ca5909bd274a7d989c3, State: Initialized, Role: FOLLOWER
I20260619 12:10:16.891999 10570 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5107c154374ca5909bd274a7d989c3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45495 } }
I20260619 12:10:16.892089 10570 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:10:16.892118 10570 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:10:16.892151 10570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:10:16.892849 10570 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5107c154374ca5909bd274a7d989c3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45495 } }
I20260619 12:10:16.892930 10570 leader_election.cc:302] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd5107c154374ca5909bd274a7d989c3; no voters:
I20260619 12:10:16.893060 10570 leader_election.cc:288] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:10:16.893149 10575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:10:16.893324 10575 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [term 1 LEADER]: Becoming Leader. State: Replica: bd5107c154374ca5909bd274a7d989c3, State: Running, Role: LEADER
I20260619 12:10:16.893342 10570 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:10:16.893453 10575 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5107c154374ca5909bd274a7d989c3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45495 } }
I20260619 12:10:16.893817 10576 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd5107c154374ca5909bd274a7d989c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5107c154374ca5909bd274a7d989c3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45495 } } }
I20260619 12:10:16.893832 10577 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd5107c154374ca5909bd274a7d989c3. Latest consensus state: current_term: 1 leader_uuid: "bd5107c154374ca5909bd274a7d989c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5107c154374ca5909bd274a7d989c3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45495 } } }
I20260619 12:10:16.894021 10577 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:16.894011 10576 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:16.894527 10585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:10:16.895001 10585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:10:16.896473 10585 catalog_manager.cc:1347] Generated new cluster ID: d1a0c7ecd33f48fb806cad42aa977169
I20260619 12:10:16.896528 10585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:10:16.905102 10585 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:10:16.905272 10585 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:10:16.911665 10585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd5107c154374ca5909bd274a7d989c3: Generated new TSK 0
I20260619 12:10:16.911859 10585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:10:16.917164 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45495
--builtin_ntp_servers=127.0.201.20:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:10:17.013135 10594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:17.013343 10594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:17.013365 10594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:17.014582 10594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:17.014626 10594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:17.014652 10594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:10:17.016155 10594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:17.016405 10594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:17.016695 10594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:17.022321 10599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:17.022307 10602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:17.022583 10594 server_base.cc:1034] running on GCE node
W20260619 12:10:17.022593 10600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:17.022969 10594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:17.023312 10594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:17.024499 10594 hybrid_clock.cc:648] HybridClock initialized: now 1781871017024509 us; error 59 us; skew 500 ppm
I20260619 12:10:17.025941 10594 webserver.cc:460] Webserver started at http://127.0.201.1:43871/ using document root <none> and password file <none>
I20260619 12:10:17.026194 10594 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:17.026249 10594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:17.026363 10594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:17.027714 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "8ed93a3073614eea8bbad4ae299a311e"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "41ab6e2747b79c887cec12105f5e5a12"
server_key_iv: "594dcc2cffd4affa6875527bebad16d8"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.028110 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "8ed93a3073614eea8bbad4ae299a311e"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "41ab6e2747b79c887cec12105f5e5a12"
server_key_iv: "594dcc2cffd4affa6875527bebad16d8"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.029695 10594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:10:17.030603 10608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.030864 10594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:17.030952 10594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "8ed93a3073614eea8bbad4ae299a311e"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "41ab6e2747b79c887cec12105f5e5a12"
server_key_iv: "594dcc2cffd4affa6875527bebad16d8"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.031026 10594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:17.035893 10594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:17.036247 10594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:17.036396 10594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:17.036604 10594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:17.036957 10594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:17.037009 10594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.037037 10594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:17.037063 10594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.043689 10594 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:46069
I20260619 12:10:17.043754 10721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:46069 every 8 connection(s)
I20260619 12:10:17.044064 10594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:10:17.046964 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:17.044781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:53415 (local address 127.0.201.62:45495)
0619 12:10:17.044847 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:17.044850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:17.045079 (+ 229us) server_negotiation.cc:407] Connection header received
0619 12:10:17.045518 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:17.045519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:17.045526 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:17.045603 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:17.046746 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:17.046748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:17.046748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:17.046752 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:17.046769 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:17.046773 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:17.046832 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:17.046885 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:10:17.046899 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:10:17.047256 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:17.044753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45495 (local address 127.0.201.1:53415)
0619 12:10:17.044944 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:17.044982 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:17.045425 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:17.045659 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:17.045662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:17.045669 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:17.046601 (+ 932us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:10:17.046602 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:17.046627 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:10:17.046654 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:17.046836 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:17.046838 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:17.046840 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:10:17.046892 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:10:17.046939 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260619 12:10:17.047750 10722 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45495
I20260619 12:10:17.047835 10722 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:17.047971 10722 heartbeater.cc:511] Master 127.0.201.62:45495 requested a full tablet report, sending...
I20260619 12:10:17.048465 10534 ts_manager.cc:194] Registered new tserver with Master: 8ed93a3073614eea8bbad4ae299a311e (127.0.201.1:46069)
I20260619 12:10:17.052664 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10594
I20260619 12:10:17.052745 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:10:17.052896 804 external_mini_cluster.cc:1428] Setting key 6b81440d6d9db6a256c6383a75747038
I20260619 12:10:17.053751 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45495
--builtin_ntp_servers=127.0.201.20:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:10:17.132893 10725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:17.133107 10725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:17.133128 10725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:17.134248 10725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:17.134285 10725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:17.134310 10725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:10:17.135682 10725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:17.135860 10725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:17.136164 10725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:17.141038 10731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:17.141090 10733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:17.141080 10725 server_base.cc:1034] running on GCE node
W20260619 12:10:17.141054 10730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:17.141405 10725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:17.141691 10725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:17.142853 10725 hybrid_clock.cc:648] HybridClock initialized: now 1781871017142839 us; error 28 us; skew 500 ppm
I20260619 12:10:17.144147 10725 webserver.cc:460] Webserver started at http://127.0.201.2:34499/ using document root <none> and password file <none>
I20260619 12:10:17.144399 10725 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:17.144470 10725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:17.144620 10725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:17.145659 10725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "aa2321b72ad4475181b422b016319a80"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "0b0f3aacd9b9b8449c1f0ada11e3534f"
server_key_iv: "6429732dccb992886e906b0d2835cfd7"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.146018 10725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "aa2321b72ad4475181b422b016319a80"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "0b0f3aacd9b9b8449c1f0ada11e3534f"
server_key_iv: "6429732dccb992886e906b0d2835cfd7"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.147353 10725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:10:17.148100 10739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.148268 10725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:17.148365 10725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "aa2321b72ad4475181b422b016319a80"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "0b0f3aacd9b9b8449c1f0ada11e3534f"
server_key_iv: "6429732dccb992886e906b0d2835cfd7"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.148479 10725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:17.151860 10725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:17.152119 10725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:17.152261 10725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:17.152465 10725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:17.152765 10725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:17.152819 10725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.152858 10725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:17.152884 10725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.158366 10725 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:35223
I20260619 12:10:17.158463 10852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:35223 every 8 connection(s)
I20260619 12:10:17.158740 10725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:10:17.161662 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:17.159505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:35847 (local address 127.0.201.62:45495)
0619 12:10:17.159580 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:17.159583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:17.159762 (+ 179us) server_negotiation.cc:407] Connection header received
0619 12:10:17.160195 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:17.160197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:17.160202 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:17.160265 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:17.161405 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:17.161406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:17.161406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:17.161411 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:17.161425 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:17.161428 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:17.161483 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:17.161598 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 12:10:17.161610 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:10:17.161818 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:17.159463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45495 (local address 127.0.201.2:35847)
0619 12:10:17.159637 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:17.159675 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:17.160114 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:17.160344 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:17.160348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:17.160355 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:17.161268 (+ 913us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:10:17.161268 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:17.161293 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:10:17.161318 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:17.161491 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:17.161513 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:17.161516 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:10:17.161576 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:10:17.161621 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260619 12:10:17.162294 10853 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45495
I20260619 12:10:17.162398 10853 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:17.162513 10853 heartbeater.cc:511] Master 127.0.201.62:45495 requested a full tablet report, sending...
I20260619 12:10:17.162814 10534 ts_manager.cc:194] Registered new tserver with Master: aa2321b72ad4475181b422b016319a80 (127.0.201.2:35223)
I20260619 12:10:17.167973 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10725
I20260619 12:10:17.168051 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:10:17.168202 804 external_mini_cluster.cc:1428] Setting key 21251086f393926eb63520f03bc97965
I20260619 12:10:17.169049 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45495
--builtin_ntp_servers=127.0.201.20:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:10:17.248118 10856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:17.248338 10856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:17.248375 10856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:17.249529 10856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:17.249593 10856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:17.249637 10856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:10:17.251027 10856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:17.251296 10856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:17.251590 10856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:17.256604 10861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:17.256747 10864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:17.256909 10856 server_base.cc:1034] running on GCE node
W20260619 12:10:17.256616 10862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:17.257136 10856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:17.257378 10856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:17.258550 10856 hybrid_clock.cc:648] HybridClock initialized: now 1781871017258545 us; error 42 us; skew 500 ppm
I20260619 12:10:17.259814 10856 webserver.cc:460] Webserver started at http://127.0.201.3:32907/ using document root <none> and password file <none>
I20260619 12:10:17.260059 10856 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:17.260128 10856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:17.260278 10856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:17.261307 10856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "d3067f1029bf4335b3e60d5700d71629"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "0fbe08d1d11f0fde77a9571222328859"
server_key_iv: "45122cc91a355b2b8d5159b4c3058bb0"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.261670 10856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "d3067f1029bf4335b3e60d5700d71629"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "0fbe08d1d11f0fde77a9571222328859"
server_key_iv: "45122cc91a355b2b8d5159b4c3058bb0"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.262912 10856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:10:17.263693 10870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.263862 10856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.263958 10856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "d3067f1029bf4335b3e60d5700d71629"
format_stamp: "Formatted at 2026-06-19 12:10:17 on dist-test-slave-c910"
server_key: "0fbe08d1d11f0fde77a9571222328859"
server_key_iv: "45122cc91a355b2b8d5159b4c3058bb0"
server_key_version: "encryptionkey@0"
I20260619 12:10:17.264039 10856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:17.267325 10856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:17.267580 10856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:17.267717 10856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:17.267912 10856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:17.268200 10856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:17.268252 10856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.268290 10856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:17.268316 10856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:17.274001 10856 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:36357
I20260619 12:10:17.274045 10983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:36357 every 8 connection(s)
I20260619 12:10:17.274354 10856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:10:17.277240 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:17.275009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:44345 (local address 127.0.201.62:45495)
0619 12:10:17.275070 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:17.275073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:17.275295 (+ 222us) server_negotiation.cc:407] Connection header received
0619 12:10:17.275764 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:17.275765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:17.275770 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:17.275829 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:17.277010 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:17.277013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:17.277013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:10:17.277017 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:17.277032 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:10:17.277036 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:17.277089 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:17.277174 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 12:10:17.277186 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:10:17.277544 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:17.274993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45495 (local address 127.0.201.3:44345)
0619 12:10:17.275162 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:17.275219 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:17.275674 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:17.275884 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:17.275887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:17.275900 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:17.276841 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:10:17.276842 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:17.276871 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:10:17.276902 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:17.277102 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:17.277106 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:17.277108 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:10:17.277168 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:10:17.277218 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260619 12:10:17.278214 10984 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45495
I20260619 12:10:17.278328 10984 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:17.278512 10984 heartbeater.cc:511] Master 127.0.201.62:45495 requested a full tablet report, sending...
I20260619 12:10:17.278964 10534 ts_manager.cc:194] Registered new tserver with Master: d3067f1029bf4335b3e60d5700d71629 (127.0.201.3:36357)
I20260619 12:10:17.283399 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 10856
I20260619 12:10:17.283499 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:10:17.283703 804 external_mini_cluster.cc:1428] Setting key 259422fbfb3525f45d837d380818a273
I20260619 12:10:17.284578 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:10:17.286398 10573 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:10:17.285897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50528 (local address 127.0.201.62:45495)
0619 12:10:17.285969 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:17.285972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:17.286076 (+ 104us) server_negotiation.cc:407] Connection header received
0619 12:10:17.286155 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:17.286156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:17.286162 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:17.286234 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:17.286345 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50528: BlockingRecv error: recv got EOF from 127.0.0.1:50528 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:10:17.286687 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10594
I20260619 12:10:17.291571 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10725
I20260619 12:10:17.295957 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10856
I20260619 12:10:17.299809 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 10503
2026-06-19T12:10:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (542 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:10:17 dist-test-slave-c910 krb5kdc[10994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:10:17 dist-test-slave-c910 krb5kdc[10994](info): set up 2 sockets
Jun 19 12:10:17 dist-test-slave-c910 krb5kdc[10994](info): commencing operation
krb5kdc: starting...
W20260619 12:10:18.332050 804 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:10:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:10:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:10:18.443181 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36583
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:36583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:18.522982 11007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:18.523202 11007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:18.523277 11007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:18.524453 11007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:10:18.524520 11007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:18.524546 11007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:18.524578 11007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:10:18.524603 11007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:10:18.526051 11007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:36583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:18.526320 11007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:18.526543 11007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:18.531319 11012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:18.531325 11013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:18.531371 11015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:18.531646 11007 server_base.cc:1034] running on GCE node
I20260619 12:10:18.531903 11007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:18.532145 11007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:18.533284 11007 hybrid_clock.cc:648] HybridClock initialized: now 1781871018533272 us; error 29 us; skew 500 ppm
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:18.535800 11007 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:10:18.536226 11007 webserver.cc:460] Webserver started at http://127.0.201.62:45035/ using document root <none> and password file <none>
I20260619 12:10:18.536450 11007 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:18.536515 11007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:18.536660 11007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:18.537665 11007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "a7d779662c214a16b9c2b77c77d553d0"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "ebad1b2aeb9726da5caebb578338572b"
server_key_iv: "155c70e7a323a9a7268bd5bfb65a334d"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.537995 11007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "a7d779662c214a16b9c2b77c77d553d0"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "ebad1b2aeb9726da5caebb578338572b"
server_key_iv: "155c70e7a323a9a7268bd5bfb65a334d"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.539304 11007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:10:18.540021 11022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.540182 11007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:18.540277 11007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "a7d779662c214a16b9c2b77c77d553d0"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "ebad1b2aeb9726da5caebb578338572b"
server_key_iv: "155c70e7a323a9a7268bd5bfb65a334d"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.540359 11007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:18.561620 11007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:18.563997 11007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:18.564177 11007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:18.568038 11007 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:36583
I20260619 12:10:18.568073 11074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:36583 every 8 connection(s)
I20260619 12:10:18.568420 11007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:10:18.569371 11075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:10:18.571583 11075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0: Bootstrap starting.
I20260619 12:10:18.572139 11075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0: Neither blocks nor log segments found. Creating new log.
I20260619 12:10:18.572432 11075 log.cc:824] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:18.573143 11075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0: No bootstrap required, opened a new log
I20260619 12:10:18.574738 11075 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d779662c214a16b9c2b77c77d553d0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36583 } }
I20260619 12:10:18.574872 11075 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:10:18.574932 11075 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d779662c214a16b9c2b77c77d553d0, State: Initialized, Role: FOLLOWER
I20260619 12:10:18.575078 11075 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d779662c214a16b9c2b77c77d553d0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36583 } }
I20260619 12:10:18.575136 11075 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:10:18.575157 11075 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:10:18.575245 11075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:10:18.576025 11075 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d779662c214a16b9c2b77c77d553d0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36583 } }
I20260619 12:10:18.576138 11075 leader_election.cc:302] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7d779662c214a16b9c2b77c77d553d0; no voters:
I20260619 12:10:18.576333 11075 leader_election.cc:288] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:10:18.576356 11078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:10:18.576485 11078 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [term 1 LEADER]: Becoming Leader. State: Replica: a7d779662c214a16b9c2b77c77d553d0, State: Running, Role: LEADER
I20260619 12:10:18.576592 11075 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:10:18.576640 11078 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d779662c214a16b9c2b77c77d553d0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36583 } }
I20260619 12:10:18.577009 11079 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7d779662c214a16b9c2b77c77d553d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d779662c214a16b9c2b77c77d553d0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36583 } } }
I20260619 12:10:18.577121 11079 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:18.577018 11080 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7d779662c214a16b9c2b77c77d553d0. Latest consensus state: current_term: 1 leader_uuid: "a7d779662c214a16b9c2b77c77d553d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d779662c214a16b9c2b77c77d553d0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36583 } } }
I20260619 12:10:18.577243 11080 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0 [sys.catalog]: This master's current role is: LEADER
I20260619 12:10:18.577344 11087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:10:18.577545 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 11007
I20260619 12:10:18.577636 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:10:18.577821 804 external_mini_cluster.cc:1428] Setting key c1873100c1bd0cf07684917da9127d01
I20260619 12:10:18.577883 11087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:10:18.579578 11087 catalog_manager.cc:1347] Generated new cluster ID: 3a562a4d913541139a13752c9b5cfaed
I20260619 12:10:18.579758 11087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:18.587244 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:18.578391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.0.201.62:36583)
0619 12:10:18.578558 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:18.578567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:18.578620 (+ 53us) server_negotiation.cc:407] Connection header received
0619 12:10:18.578724 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:18.578729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:18.578771 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:18.578908 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:18.579657 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.580608 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.581349 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.581763 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.581881 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:18.584123 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:18.584127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:18.584127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:18.584138 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:18.585749 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:18.586111 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:18.586113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:18.586114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:18.586204 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:18.586406 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:18.586407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:18.586407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:18.586629 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:18.586790 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:18.586880 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 12:10:18.587024 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260619 12:10:18.595856 11087 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:10:18.596338 11087 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:10:18.600075 11087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0: Generated new TSK 0
I20260619 12:10:18.600245 11087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:10:18.631136 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36583
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:18.733040 11101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:18.733228 11101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:18.733251 11101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:18.734378 11101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:18.734418 11101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:18.734444 11101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:10:18.735936 11101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:18.736166 11101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:18.736392 11101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:18.741143 11109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:18.741150 11107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:18.741154 11106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:18.741328 11101 server_base.cc:1034] running on GCE node
I20260619 12:10:18.741515 11101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:18.741748 11101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:18.742913 11101 hybrid_clock.cc:648] HybridClock initialized: now 1781871018742908 us; error 36 us; skew 500 ppm
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:18.745428 11101 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:10:18.745842 11101 webserver.cc:460] Webserver started at http://127.0.201.1:35309/ using document root <none> and password file <none>
I20260619 12:10:18.746073 11101 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:18.746140 11101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:18.746271 11101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:18.747335 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "3079b7edd9eb4d58bfde10ae39eb82b6"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "4c302ce7e13ac4a84711d34f63ac9e2f"
server_key_iv: "df2fbdac70cb49080210e7e67f255e60"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.747679 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "3079b7edd9eb4d58bfde10ae39eb82b6"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "4c302ce7e13ac4a84711d34f63ac9e2f"
server_key_iv: "df2fbdac70cb49080210e7e67f255e60"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.748936 11101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:10:18.749783 11116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.749963 11101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:18.750056 11101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "3079b7edd9eb4d58bfde10ae39eb82b6"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "4c302ce7e13ac4a84711d34f63ac9e2f"
server_key_iv: "df2fbdac70cb49080210e7e67f255e60"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.750124 11101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:18.769362 11101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:18.771036 11101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:18.771214 11101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:18.771426 11101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:18.771708 11101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:18.771760 11101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.771799 11101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:18.771837 11101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.777498 11101 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:43955
I20260619 12:10:18.777552 11229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:43955 every 8 connection(s)
I20260619 12:10:18.777869 11101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:18.786082 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:18.778568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:44785 (local address 127.0.201.62:36583)
0619 12:10:18.778650 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:18.778652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:18.778830 (+ 178us) server_negotiation.cc:407] Connection header received
0619 12:10:18.779318 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:18.779319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:18.779359 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:18.779411 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:18.780241 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.780745 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.781657 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.781847 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.781898 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:18.783829 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:18.783831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:18.783831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:18.783837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:18.785011 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:18.785359 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:18.785360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:18.785360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:18.785408 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:18.785614 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:18.785614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:18.785614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:18.785705 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:18.785753 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:18.785961 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 12:10:18.786021 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:10:18.786212 11232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:18.778520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36583 (local address 127.0.201.1:44785)
0619 12:10:18.778688 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:18.778720 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:18.779186 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:18.779460 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:18.779463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:18.779986 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:18.780157 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:18.780162 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.780822 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.780822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:18.781585 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:18.781586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.781914 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.781915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:18.782001 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:18.782415 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:18.782422 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:18.783749 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:18.785076 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:18.785080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:18.785082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:18.785280 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:18.785464 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:18.785464 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:18.785464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:18.785562 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:18.785756 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:18.785759 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:18.785860 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 12:10:18.785937 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:10:18.786038 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260619 12:10:18.786451 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 11101
I20260619 12:10:18.786556 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:10:18.786743 804 external_mini_cluster.cc:1428] Setting key 661a06cdcb10ee826d3bf9654986b405
I20260619 12:10:18.786749 11230 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36583
I20260619 12:10:18.786828 11230 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:18.787022 11230 heartbeater.cc:511] Master 127.0.201.62:36583 requested a full tablet report, sending...
I20260619 12:10:18.787569 11039 ts_manager.cc:194] Registered new tserver with Master: 3079b7edd9eb4d58bfde10ae39eb82b6 (127.0.201.1:43955)
I20260619 12:10:18.788168 11039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:44785
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:10:18.807852 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36583
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:18.888182 11235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:18.888386 11235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:18.888411 11235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:18.889467 11235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:18.889539 11235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:18.889568 11235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:10:18.890914 11235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:18.891094 11235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:18.891291 11235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:18.896174 11243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:18.896279 11235 server_base.cc:1034] running on GCE node
W20260619 12:10:18.896186 11241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:18.896204 11240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:18.896600 11235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:18.896876 11235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:18.898065 11235 hybrid_clock.cc:648] HybridClock initialized: now 1781871018898054 us; error 26 us; skew 500 ppm
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:18.900539 11235 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:10:18.900997 11235 webserver.cc:460] Webserver started at http://127.0.201.2:44749/ using document root <none> and password file <none>
I20260619 12:10:18.901222 11235 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:18.901290 11235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:18.901433 11235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:18.902503 11235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "27216c43311e40099886ab021fee40f9"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "6050740e8d587febd7b21ea8398267f7"
server_key_iv: "e7392f679304c5d05c0a9f18e18f0aa4"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.902908 11235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "27216c43311e40099886ab021fee40f9"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "6050740e8d587febd7b21ea8398267f7"
server_key_iv: "e7392f679304c5d05c0a9f18e18f0aa4"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.904346 11235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 12:10:18.905092 11250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.905264 11235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:10:18.905359 11235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "27216c43311e40099886ab021fee40f9"
format_stamp: "Formatted at 2026-06-19 12:10:18 on dist-test-slave-c910"
server_key: "6050740e8d587febd7b21ea8398267f7"
server_key_iv: "e7392f679304c5d05c0a9f18e18f0aa4"
server_key_version: "encryptionkey@0"
I20260619 12:10:18.905472 11235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:18.917762 11235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:18.919706 11235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:18.919862 11235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:18.920073 11235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:18.920419 11235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:18.920483 11235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.920516 11235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:18.920547 11235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:18.926517 11235 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:39469
I20260619 12:10:18.926553 11363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:39469 every 8 connection(s)
I20260619 12:10:18.926862 11235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:10:18.931890 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 11235
I20260619 12:10:18.931967 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:10:18.932156 804 external_mini_cluster.cc:1428] Setting key 4a7a5e24a77255c1fd98348213a84ddd
Jun 19 12:10:18 dist-test-slave-c910 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:18.936154 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:18.927539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:34091 (local address 127.0.201.62:36583)
0619 12:10:18.927600 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:18.927602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:18.927802 (+ 200us) server_negotiation.cc:407] Connection header received
0619 12:10:18.928285 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:18.928286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:18.928303 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:18.928397 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:18.929265 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.929763 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.930721 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.930917 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.930975 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:18.933236 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:18.933237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:18.933238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:18.933244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:18.934668 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:18.935153 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:18.935154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:18.935154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:18.935244 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:18.935525 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:18.935526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:18.935526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:18.935639 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:18.935715 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:18.935989 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 12:10:18.936067 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:10:18.936344 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:18.927516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36583 (local address 127.0.201.2:34091)
0619 12:10:18.927670 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:18.927709 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:18.928197 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:18.928454 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:18.928457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:18.928993 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:18.929168 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:18.929172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.929843 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.929844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:18.930640 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:18.930642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:18.930991 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:18.930991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:18.931089 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:18.931546 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:18.931554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:18.933149 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:18.934749 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:18.934755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:18.934757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:18.935087 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:18.935326 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:18.935326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:18.935327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:18.935448 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:18.935706 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:18.935712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:18.935846 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 12:10:18.935953 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:10:18.936101 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260619 12:10:18.936749 11364 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36583
I20260619 12:10:18.936832 11364 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:18.937011 11364 heartbeater.cc:511] Master 127.0.201.62:36583 requested a full tablet report, sending...
I20260619 12:10:18.937389 11039 ts_manager.cc:194] Registered new tserver with Master: 27216c43311e40099886ab021fee40f9 (127.0.201.2:39469)
I20260619 12:10:18.937871 11039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:34091
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:10:18.953533 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36583
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:10:19.032382 11369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:10:19.032573 11369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:10:19.032594 11369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:10:19.033648 11369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:10:19.033682 11369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:10:19.033707 11369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:10:19.035071 11369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:10:19.035315 11369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:10:19.035583 11369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:10:19.040429 11377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:19.040508 11369 server_base.cc:1034] running on GCE node
W20260619 12:10:19.040403 11375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:10:19.040462 11374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:10:19.040791 11369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:10:19.041038 11369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:10:19.042200 11369 hybrid_clock.cc:648] HybridClock initialized: now 1781871019042186 us; error 46 us; skew 500 ppm
Jun 19 12:10:19 dist-test-slave-c910 krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:10:19.044739 11369 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:10:19.045153 11369 webserver.cc:460] Webserver started at http://127.0.201.3:37087/ using document root <none> and password file <none>
I20260619 12:10:19.045372 11369 fs_manager.cc:359] Metadata directory not provided
I20260619 12:10:19.045414 11369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:10:19.045554 11369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:10:19.046545 11369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "610c5a56a3094567b37c4c35fbcb8d8a"
format_stamp: "Formatted at 2026-06-19 12:10:19 on dist-test-slave-c910"
server_key: "ccc15b95793d4d26242b62ee52e0616e"
server_key_iv: "22f1591f7862da3203c124f0c087e66e"
server_key_version: "encryptionkey@0"
I20260619 12:10:19.046887 11369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "610c5a56a3094567b37c4c35fbcb8d8a"
format_stamp: "Formatted at 2026-06-19 12:10:19 on dist-test-slave-c910"
server_key: "ccc15b95793d4d26242b62ee52e0616e"
server_key_iv: "22f1591f7862da3203c124f0c087e66e"
server_key_version: "encryptionkey@0"
I20260619 12:10:19.048226 11369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:10:19.048887 11384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:19.049054 11369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:10:19.049134 11369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "610c5a56a3094567b37c4c35fbcb8d8a"
format_stamp: "Formatted at 2026-06-19 12:10:19 on dist-test-slave-c910"
server_key: "ccc15b95793d4d26242b62ee52e0616e"
server_key_iv: "22f1591f7862da3203c124f0c087e66e"
server_key_version: "encryptionkey@0"
I20260619 12:10:19.049183 11369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:10:19.060266 11369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:10:19.061962 11369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:10:19.062062 11369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:10:19.062222 11369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:10:19.062528 11369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:10:19.062561 11369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:19.062582 11369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:10:19.062628 11369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:10:19.068409 11369 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:35107
I20260619 12:10:19.068475 11497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:35107 every 8 connection(s)
I20260619 12:10:19.068768 11369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:10:19 dist-test-slave-c910 krb5kdc[10994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:10:19.077047 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.069398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:41969 (local address 127.0.201.62:36583)
0619 12:10:19.069438 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.069440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.069644 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:10:19.070087 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.070088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.070097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.070145 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:19.071031 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.071578 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.072478 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.072651 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.072698 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.074615 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:19.074616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:19.074617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:19.074623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:19.075867 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:19.076215 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:19.076216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:19.076216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:19.076264 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:19.076456 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:19.076457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:19.076457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:19.076590 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:19.076655 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.076887 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.076946 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:10:19.077255 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.069382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36583 (local address 127.0.201.3:41969)
0619 12:10:19.069518 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.069571 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:19.070007 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:19.070193 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:19.070196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:19.070776 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:10:19.070957 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.070962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.071648 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.071648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.072407 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.072409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.072716 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.072717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.072808 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.073299 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:10:19.073306 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:10:19.074537 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:10:19.075935 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:19.075940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:19.075941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:19.076152 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:19.076319 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:10:19.076320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:10:19.076320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:10:19.076404 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:10:19.076660 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:10:19.076664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:10:19.076769 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 12:10:19.076851 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:10:19.076957 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":84,"threads_started":1}
I20260619 12:10:19.077814 11498 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36583
I20260619 12:10:19.077859 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 11369
I20260619 12:10:19.077915 11498 heartbeater.cc:464] Registering TS with master...
I20260619 12:10:19.077960 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:10:19.078102 11498 heartbeater.cc:511] Master 127.0.201.62:36583 requested a full tablet report, sending...
I20260619 12:10:19.078189 804 external_mini_cluster.cc:1428] Setting key e6eb71bf5317670c0e0148c478ca4b44
I20260619 12:10:19.078459 11039 ts_manager.cc:194] Registered new tserver with Master: 610c5a56a3094567b37c4c35fbcb8d8a (127.0.201.3:35107)
I20260619 12:10:19.078903 11039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:41969
I20260619 12:10:19.078995 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:10:19.085335 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.080207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.0.201.62:36583)
0619 12:10:19.080241 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.080243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.080326 (+ 83us) server_negotiation.cc:407] Connection header received
0619 12:10:19.080368 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.080369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.080378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.080426 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:10:19.080928 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.081360 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.082020 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.082183 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.082228 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.083397 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:10:19.083397 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:10:19.083398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:10:19.083403 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:10:19.084480 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:19.084766 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:19.084766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:19.084767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:19.084815 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:10:19.084957 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:10:19.084957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:10:19.084957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:10:19.085068 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:10:19.085135 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.085226 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.085277 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:10:19.086740 11039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:10:19.088419 11039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:10:19.094550 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.090408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:35107 (local address 127.0.0.1:50422)
0619 12:10:19.090976 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.090986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:19.091047 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:19.091302 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:19.091303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:19.091305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:10:19.091341 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.091343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.092224 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.092225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.093593 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.093596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.094266 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.094267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.094340 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.094362 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 12:10:19.094431 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:10:19.094479 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":55,"thread_start_us":64,"threads_started":1}
I20260619 12:10:19.094560 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.090613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50422 (local address 127.0.201.3:35107)
0619 12:10:19.090790 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.090794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.091037 (+ 243us) server_negotiation.cc:407] Connection header received
0619 12:10:19.091115 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.091118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.091133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.091229 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:10:19.091417 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.092127 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.093679 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.094178 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.094266 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.094323 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.094449 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.094481 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260619 12:10:19.094749 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.090405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:39469 (local address 127.0.0.1:57038)
0619 12:10:19.090508 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.090545 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:19.090605 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:19.090968 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:19.090970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:19.090972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:10:19.091034 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.091036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.092315 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.092316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.093593 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.093596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.094503 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.094504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.094580 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.094585 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:10:19.094616 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 12:10:19.094662 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":46,"thread_start_us":57,"threads_started":1}
I20260619 12:10:19.095082 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.090648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.0.201.2:39469)
0619 12:10:19.090807 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.090811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.090820 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:10:19.090855 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.090857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.090877 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.090965 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:10:19.091119 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.092219 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.093692 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.094403 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.094845 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.094909 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.094938 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.094975 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
I20260619 12:10:19.095180 11432 tablet_service.cc:1467] Processing CreateTablet for tablet dbfe28cec7a74c8fae835f33b08d84f3 (DEFAULT_TABLE table=test-table [id=ededc9723ae14f458ee75f9ef9158131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:10:19.095435 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.090821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.0.201.1:43955)
0619 12:10:19.090964 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.090969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.091240 (+ 271us) server_negotiation.cc:407] Connection header received
0619 12:10:19.091332 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.091334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.091349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.091415 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:10:19.091618 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.092650 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.093722 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.094403 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.094474 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.094538 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.095299 (+ 761us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.095355 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260619 12:10:19.095585 11432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfe28cec7a74c8fae835f33b08d84f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:10:19.095721 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.090431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:43955 (local address 127.0.0.1:49902)
0619 12:10:19.091123 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.091132 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:19.091174 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:19.091491 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:19.091491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:19.091493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:10:19.091533 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.091534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.092757 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.092758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.093631 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.093654 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.094567 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.094568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.095015 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.095020 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:10:19.095071 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:10:19.095113 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":44,"threads_started":1}
I20260619 12:10:19.095744 11298 tablet_service.cc:1467] Processing CreateTablet for tablet dbfe28cec7a74c8fae835f33b08d84f3 (DEFAULT_TABLE table=test-table [id=ededc9723ae14f458ee75f9ef9158131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:10:19.096065 11298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfe28cec7a74c8fae835f33b08d84f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:10:19.096468 11164 tablet_service.cc:1467] Processing CreateTablet for tablet dbfe28cec7a74c8fae835f33b08d84f3 (DEFAULT_TABLE table=test-table [id=ededc9723ae14f458ee75f9ef9158131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:10:19.096762 11164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfe28cec7a74c8fae835f33b08d84f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:10:19.097826 11512 tablet_bootstrap.cc:492] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Bootstrap starting.
I20260619 12:10:19.098467 11512 tablet_bootstrap.cc:654] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Neither blocks nor log segments found. Creating new log.
I20260619 12:10:19.098853 11512 log.cc:824] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:19.099144 11514 tablet_bootstrap.cc:492] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Bootstrap starting.
I20260619 12:10:19.099510 11512 tablet_bootstrap.cc:492] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: No bootstrap required, opened a new log
I20260619 12:10:19.099552 11512 ts_tablet_manager.cc:1397] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:10:19.099743 11515 tablet_bootstrap.cc:492] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: Bootstrap starting.
I20260619 12:10:19.100301 11515 tablet_bootstrap.cc:654] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: Neither blocks nor log segments found. Creating new log.
I20260619 12:10:19.100893 11515 log.cc:824] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:19.101159 11512 raft_consensus.cc:348] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.101336 11512 raft_consensus.cc:374] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:10:19.101392 11512 raft_consensus.cc:729] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610c5a56a3094567b37c4c35fbcb8d8a, State: Initialized, Role: FOLLOWER
I20260619 12:10:19.101504 11512 consensus_queue.cc:260] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.101585 11515 tablet_bootstrap.cc:492] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: No bootstrap required, opened a new log
I20260619 12:10:19.101665 11515 ts_tablet_manager.cc:1397] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:10:19.101780 11512 ts_tablet_manager.cc:1428] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:10:19.101806 11498 heartbeater.cc:503] Master 127.0.201.62:36583 was elected leader, sending a full tablet report...
I20260619 12:10:19.102412 11514 tablet_bootstrap.cc:654] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Neither blocks nor log segments found. Creating new log.
I20260619 12:10:19.102981 11514 log.cc:824] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:10:19.103309 11515 raft_consensus.cc:348] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.103479 11515 raft_consensus.cc:374] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:10:19.103528 11515 raft_consensus.cc:729] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3079b7edd9eb4d58bfde10ae39eb82b6, State: Initialized, Role: FOLLOWER
I20260619 12:10:19.103631 11515 consensus_queue.cc:260] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.103749 11514 tablet_bootstrap.cc:492] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: No bootstrap required, opened a new log
I20260619 12:10:19.103813 11514 ts_tablet_manager.cc:1397] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 12:10:19.103902 11515 ts_tablet_manager.cc:1428] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:10:19.103940 11230 heartbeater.cc:503] Master 127.0.201.62:36583 was elected leader, sending a full tablet report...
I20260619 12:10:19.105420 11514 raft_consensus.cc:348] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.105571 11514 raft_consensus.cc:374] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:10:19.105615 11514 raft_consensus.cc:729] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27216c43311e40099886ab021fee40f9, State: Initialized, Role: FOLLOWER
I20260619 12:10:19.105720 11514 consensus_queue.cc:260] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.105958 11514 ts_tablet_manager.cc:1428] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:10:19.106029 11364 heartbeater.cc:503] Master 127.0.201.62:36583 was elected leader, sending a full tablet report...
W20260619 12:10:19.177433 11365 tablet.cc:2369] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:10:19.278627 11231 tablet.cc:2369] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:10:19.299346 11519 raft_consensus.cc:482] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:10:19.299463 11519 raft_consensus.cc:504] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.299791 11519 leader_election.cc:288] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27216c43311e40099886ab021fee40f9 (127.0.201.2:39469), 610c5a56a3094567b37c4c35fbcb8d8a (127.0.201.3:35107)
I20260619 12:10:19.302765 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.300071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:45291 (local address 127.0.201.3:35107)
0619 12:10:19.300143 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.300145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.300223 (+ 78us) server_negotiation.cc:407] Connection header received
0619 12:10:19.300314 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.300316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.300326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.300411 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:10:19.300579 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.301068 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.302050 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.302418 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.302500 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.302510 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.302666 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.302702 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:10:19.302783 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.299979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:35107 (local address 127.0.201.1:45291)
0619 12:10:19.300139 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.300151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:19.300238 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:19.300414 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:19.300415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:19.300416 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:10:19.300491 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.300493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.301177 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.301177 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.301956 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.301957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.302499 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.302500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.302581 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.302585 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:10:19.302657 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:10:19.302695 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":24,"spinlock_wait_cycles":12544,"thread_start_us":95,"threads_started":1}
I20260619 12:10:19.302845 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.300059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:35677 (local address 127.0.201.2:39469)
0619 12:10:19.300107 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.300110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.300213 (+ 103us) server_negotiation.cc:407] Connection header received
0619 12:10:19.300287 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.300288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.300298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.300463 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:10:19.300634 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.301107 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.302111 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.302491 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.302584 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.302593 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.302740 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.302777 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:10:19.303174 11452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe28cec7a74c8fae835f33b08d84f3" candidate_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" is_pre_election: true
I20260619 12:10:19.303329 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.299979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:39469 (local address 127.0.201.1:35677)
0619 12:10:19.300134 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.300144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:10:19.300230 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:10:19.300469 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:10:19.300470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:10:19.300472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:10:19.300562 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.300563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.301219 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.301220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.302024 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:10:19.302026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.302587 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.302588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:10:19.302660 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.302663 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:10:19.302724 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:10:19.302763 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260619 12:10:19.303381 11452 raft_consensus.cc:2454] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3079b7edd9eb4d58bfde10ae39eb82b6 in term 0.
I20260619 12:10:19.303596 11118 leader_election.cc:302] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3079b7edd9eb4d58bfde10ae39eb82b6, 610c5a56a3094567b37c4c35fbcb8d8a; no voters:
I20260619 12:10:19.303622 11318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe28cec7a74c8fae835f33b08d84f3" candidate_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27216c43311e40099886ab021fee40f9" is_pre_election: true
I20260619 12:10:19.303740 11519 raft_consensus.cc:2781] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:10:19.303752 11318 raft_consensus.cc:2454] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3079b7edd9eb4d58bfde10ae39eb82b6 in term 0.
I20260619 12:10:19.303797 11519 raft_consensus.cc:482] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:10:19.303833 11519 raft_consensus.cc:3037] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:10:19.304528 11519 raft_consensus.cc:504] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.304672 11519 leader_election.cc:288] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [CANDIDATE]: Term 1 election: Requested vote from peers 27216c43311e40099886ab021fee40f9 (127.0.201.2:39469), 610c5a56a3094567b37c4c35fbcb8d8a (127.0.201.3:35107)
I20260619 12:10:19.304808 11452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe28cec7a74c8fae835f33b08d84f3" candidate_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610c5a56a3094567b37c4c35fbcb8d8a"
I20260619 12:10:19.304888 11452 raft_consensus.cc:3037] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:10:19.304893 11318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe28cec7a74c8fae835f33b08d84f3" candidate_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27216c43311e40099886ab021fee40f9"
I20260619 12:10:19.304991 11318 raft_consensus.cc:3037] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:10:19.305581 11452 raft_consensus.cc:2454] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3079b7edd9eb4d58bfde10ae39eb82b6 in term 1.
I20260619 12:10:19.305691 11318 raft_consensus.cc:2454] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3079b7edd9eb4d58bfde10ae39eb82b6 in term 1.
I20260619 12:10:19.305748 11118 leader_election.cc:302] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3079b7edd9eb4d58bfde10ae39eb82b6, 610c5a56a3094567b37c4c35fbcb8d8a; no voters:
I20260619 12:10:19.305835 11519 raft_consensus.cc:2781] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:10:19.305966 11519 raft_consensus.cc:686] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 1 LEADER]: Becoming Leader. State: Replica: 3079b7edd9eb4d58bfde10ae39eb82b6, State: Running, Role: LEADER
I20260619 12:10:19.306066 11519 consensus_queue.cc:237] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } }
I20260619 12:10:19.306846 11038 catalog_manager.cc:5495] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3079b7edd9eb4d58bfde10ae39eb82b6 (127.0.201.1). New cstate: current_term: 1 leader_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3079b7edd9eb4d58bfde10ae39eb82b6" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43955 } health_report { overall_health: HEALTHY } } }
W20260619 12:10:19.319502 11499 tablet.cc:2369] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:10:19.321698 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.319242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.0.201.1:43955)
0619 12:10:19.319286 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:10:19.319288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:10:19.319300 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:10:19.319399 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:10:19.319400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:10:19.319421 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:10:19.319490 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:10:19.319670 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.320160 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.320926 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:10:19.321102 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:10:19.321229 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:10:19.321437 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:10:19.321535 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:10:19.321596 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:10:19.321631 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:10:19.321647 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:10:19.323369 11452 raft_consensus.cc:1261] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 1 FOLLOWER]: Refusing update from remote peer 3079b7edd9eb4d58bfde10ae39eb82b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:10:19.323372 11318 raft_consensus.cc:1261] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 1 FOLLOWER]: Refusing update from remote peer 3079b7edd9eb4d58bfde10ae39eb82b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:10:19.323693 11523 consensus_queue.cc:1035] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27216c43311e40099886ab021fee40f9" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:10:19.323802 11519 consensus_queue.cc:1035] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610c5a56a3094567b37c4c35fbcb8d8a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:10:19.326910 11528 mvcc.cc:205] Tried to move back new op lower bound from 7298543695145926656 to 7298543695078309888. Current Snapshot: MvccSnapshot[applied={T|T < 7298543695145926656 or (T in {7298543695145926656})}]
I20260619 12:10:19.326970 11527 mvcc.cc:205] Tried to move back new op lower bound from 7298543695145926656 to 7298543695078309888. Current Snapshot: MvccSnapshot[applied={T|T < 7298543695145926656}]
I20260619 12:10:19.328289 11038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:10:19.328414 11038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:10:19.329135 11038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7d779662c214a16b9c2b77c77d553d0: Sending DeleteTablet for 3 replicas of tablet dbfe28cec7a74c8fae835f33b08d84f3
I20260619 12:10:19.329463 11298 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfe28cec7a74c8fae835f33b08d84f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:10:19 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:57038
I20260619 12:10:19.329690 11535 tablet_replica.cc:330] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: stopping tablet replica
I20260619 12:10:19.329691 11164 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfe28cec7a74c8fae835f33b08d84f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:10:19 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49902
I20260619 12:10:19.329792 11535 raft_consensus.cc:2229] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:10:19.329897 11535 raft_consensus.cc:2258] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:10:19.330118 11536 tablet_replica.cc:330] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6: stopping tablet replica
I20260619 12:10:19.330096 11432 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfe28cec7a74c8fae835f33b08d84f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:10:19 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:50422
I20260619 12:10:19.330207 11536 raft_consensus.cc:2229] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:10:19.330286 11536 raft_consensus.cc:2258] T dbfe28cec7a74c8fae835f33b08d84f3 P 3079b7edd9eb4d58bfde10ae39eb82b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:10:19.330309 11535 ts_tablet_manager.cc:1905] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:10:19.330380 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 11101
I20260619 12:10:19.330981 11537 tablet_replica.cc:330] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: stopping tablet replica
I20260619 12:10:19.331077 11537 raft_consensus.cc:2229] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:10:19.331163 11537 raft_consensus.cc:2258] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:10:19.331624 11537 ts_tablet_manager.cc:1905] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:10:19.332407 11535 ts_tablet_manager.cc:1918] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:10:19.332492 11535 log.cc:1196] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/dbfe28cec7a74c8fae835f33b08d84f3
I20260619 12:10:19.332768 11535 ts_tablet_manager.cc:1939] T dbfe28cec7a74c8fae835f33b08d84f3 P 27216c43311e40099886ab021fee40f9: Deleting consensus metadata
I20260619 12:10:19.333140 11023 catalog_manager.cc:4841] TS 27216c43311e40099886ab021fee40f9 (127.0.201.2:39469): tablet dbfe28cec7a74c8fae835f33b08d84f3 (table test-table [id=ededc9723ae14f458ee75f9ef9158131]) successfully deleted
I20260619 12:10:19.333187 11537 ts_tablet_manager.cc:1918] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:10:19.333240 11537 log.cc:1196] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/dbfe28cec7a74c8fae835f33b08d84f3
I20260619 12:10:19.333449 11537 ts_tablet_manager.cc:1939] T dbfe28cec7a74c8fae835f33b08d84f3 P 610c5a56a3094567b37c4c35fbcb8d8a: Deleting consensus metadata
I20260619 12:10:19.333791 11024 catalog_manager.cc:4841] TS 610c5a56a3094567b37c4c35fbcb8d8a (127.0.201.3:35107): tablet dbfe28cec7a74c8fae835f33b08d84f3 (table test-table [id=ededc9723ae14f458ee75f9ef9158131]) successfully deleted
W20260619 12:10:19.335884 11026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:43955 (error 108)
I20260619 12:10:19.336156 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 11235
I20260619 12:10:19.336350 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:10:19.336240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:43955 (local address 127.0.0.1:49912)
0619 12:10:19.336283 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:10:19.336314 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 12:10:19.336665 11026 catalog_manager.cc:4584] TS 3079b7edd9eb4d58bfde10ae39eb82b6 (127.0.201.1:43955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbfe28cec7a74c8fae835f33b08d84f3: Network error: Client connection negotiation failed: client connection to 127.0.201.1:43955: connect: Connection refused (error 111)
I20260619 12:10:19.341813 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 11369
I20260619 12:10:19.346297 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 11007
2026-06-19T12:10:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2047 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:10:19 dist-test-slave-c910 krb5kdc[11541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:10:19 dist-test-slave-c910 krb5kdc[11541](info): set up 2 sockets
Jun 19 12:10:19 dist-test-slave-c910 krb5kdc[11541](info): commencing operation
krb5kdc: starting...
W20260619 12:10:20.372212 804 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:10:20 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:10:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:10:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/rangeradmin_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/rangeradmin_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/rangerlookup_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/rangerlookup_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
I20260619 12:10:20.537125 804 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq9ki8M/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/postgres -l logfile start
2026-06-19 12:10:22.831 UTC [11571] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 12:10:22.831 UTC [11571] LOG: listening on IPv4 address "127.0.201.20", port 51047
2026-06-19 12:10:22.835 UTC [11571] LOG: listening on Unix socket "/tmp/.s.PGSQL.51047"
2026-06-19 12:10:22.870 UTC [11574] LOG: database system was shut down at 2026-06-19 12:10:21 UTC
2026-06-19 12:10:22.874 UTC [11571] LOG: database system is ready to accept connections
I20260619 12:10:23.812152 804 mini_postgres.cc:96] Postgres bound to 51047
2026-06-19 12:10:23.818 UTC [11582] FATAL: database "slave" does not exist
127.0.201.20:51047 - accepting connections
I20260619 12:10:23.818563 804 mini_ranger.cc:161] Starting Ranger...
I20260619 12:10:23.832757 804 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 12:10:24.292268 804 mini_ranger.cc:87] Created ranger Postgres database
I20260619 12:10:24.292352 804 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-admin
2026-06-19 12:10:24,578 [I] DB FLAVOR :POSTGRES
2026-06-19 12:10:24,578 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:10:24,578 [I] Checking connection..
2026-06-19 12:10:24,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 12:10:24,827 [I] Checking connection passed.
2026-06-19 12:10:24,827 [I] --------- Verifying version history table ---------
2026-06-19 12:10:24,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:10:25,082 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:10:25,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:10:25,331 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:10:25,331 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 12:10:25,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 12:10:25.570 UTC [11646] WARNING: there is no transaction in progress
2026-06-19 12:10:25,576 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 12:10:25,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:10:25,812 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 12:10:25,813 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:10:25,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 12:10:26,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 12:10:26,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c910.c.gcp-upstream.internal','N') ;"
2026-06-19 12:10:26,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:10:26,719 [I] Table x_portal_user does not exist in database ranger
2026-06-19 12:10:26,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:10:26,944 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 12:10:26,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:10:27,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:10:27,422 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 12:10:27,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 12:10:28.345 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.361 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.376 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.390 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.405 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.424 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.479 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.485 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.499 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.510 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.520 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.534 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.545 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.552 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.559 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:28.566 UTC [11754] WARNING: there is no transaction in progress
2026-06-19 12:10:29,185 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 12:10:29,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 12:10:29,430 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 12:10:29,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:10:29,666 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 12:10:29,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:10:29,925 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 12:10:29,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:10:30,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:10:30,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c910.c.gcp-upstream.internal';"
2026-06-19 12:10:30,605 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 12:10:30,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 12:10:30,850 [I] DB_PATCHES have already been applied
I20260619 12:10:30.855496 804 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq9ki8M/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 12:10:30.855558 804 mini_ranger.cc:193] Using host: 127.0.201.20
I20260619 12:10:30.858506 804 mini_ranger.cc:227] Ranger admin URL: http://127.0.201.20:60439
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 12:10:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 12:10:31.986224 804 mini_ranger.cc:160] Time spent starting Ranger: real 8.168s user 0.000s sys 0.009s
Jun 19, 2026 12:10:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 12:10:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 12:10:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 12:10:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 12:10:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 12:10:52.066579 804 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/rangerkms_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/rangerkms_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.201.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.201.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 12:10:52 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871052, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 12:10:52.143282 804 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 12:10:52.151819 804 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 12:10:52.942747 804 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 12:10:52.942843 804 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms
I20260619 12:10:52.942862 804 mini_ranger_kms.cc:227] Using postgres at 127.0.201.20:51047
2026-06-19 12:10:53,155 [I] DB FLAVOR :POSTGRES
2026-06-19 12:10:53,155 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:10:53,155 [I] Checking connection
2026-06-19 12:10:53,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 12:10:53,414 [I] connection success
2026-06-19 12:10:53,414 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 12:10:53,414 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 12:10:53,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 12:10:53,714 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 12:10:53,714 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:10:53,715 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 12:10:53,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:51047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 12:10:53,975 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 12:10:54.409317 804 mini_ranger_kms.cc:310] Created kms service
I20260619 12:10:54.585079 804 mini_ranger_kms.cc:326] Created kudu user
I20260619 12:10:54.624972 804 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 12:10:54 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871052, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:10:55.001904 804 mini_ranger_kms.cc:384] Added ranger policy
I20260619 12:10:55.002028 804 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 12:10:55.002053 804 mini_ranger_kms.cc:240] Using host: 127.0.201.20
I20260619 12:10:55.005585 804 mini_ranger_kms.cc:276] Ranger KMS PID: 11934
I20260619 12:10:55.005692 804 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.201.20:56279
Jun 19, 2026 12:10:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:10:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:10:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 12:10:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 12:10:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 12:10:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 12:10:57.331785 804 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.189s user 0.000s sys 0.194s
I20260619 12:10:57.331888 804 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 12:10:57.331979 804 mini_ranger_kms.cc:401] 127.0.201.20:56279/kms/v1/keys
Jun 19 12:10:57 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871057, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.201.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 12:10:58 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871057, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.201.20@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
Jun 19 12:10:58 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871057, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.201.20@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
Jun 19 12:11:00 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:11:00.834297 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:32935
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:32935
--ranger_config_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:56279/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:00.925779 11985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:00.925976 11985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:00.925997 11985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:00.927153 11985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:00.927242 11985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:00.927299 11985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:00.927316 11985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:00.927343 11985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:00.928957 11985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.201.20:56279/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:32935
--ranger_config_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:00.929177 11985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:00.929425 11985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:00.934283 11991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:00.934336 11993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:00.934424 11990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:00.934599 11985 server_base.cc:1034] running on GCE node
I20260619 12:11:00.934828 11985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:00.935122 11985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:00.936292 11985 hybrid_clock.cc:648] HybridClock initialized: now 1781871060936278 us; error 27 us; skew 500 ppm
Jun 19 12:11:00 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871060, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:00.943710 11985 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:00.944147 11985 webserver.cc:460] Webserver started at http://127.0.201.62:45223/ using document root <none> and password file <none>
I20260619 12:11:00.944378 11985 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:00.944437 11985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:00.944590 11985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:00 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871060, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:00.994769 11985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "8fb3a509ce914d799e6ca83f11214abe"
format_stamp: "Formatted at 2026-06-19 12:11:00 on dist-test-slave-c910"
server_key: "7ae4a453307c8d083db1820a77d7ffed"
server_key_iv: "8ce57e102c0a635e1ee29668ee2f4ecc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:00.995170 11985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "8fb3a509ce914d799e6ca83f11214abe"
format_stamp: "Formatted at 2026-06-19 12:11:00 on dist-test-slave-c910"
server_key: "7ae4a453307c8d083db1820a77d7ffed"
server_key_iv: "8ce57e102c0a635e1ee29668ee2f4ecc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:00.996627 11985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:01.010125 12000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:01.010561 11985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:01.010677 11985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "8fb3a509ce914d799e6ca83f11214abe"
format_stamp: "Formatted at 2026-06-19 12:11:00 on dist-test-slave-c910"
server_key: "7ae4a453307c8d083db1820a77d7ffed"
server_key_iv: "8ce57e102c0a635e1ee29668ee2f4ecc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:01.010790 11985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:01.035048 11985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:01.037045 11985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:01.037214 11985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:01.042120 11985 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:32935
I20260619 12:11:01.042142 12052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:32935 every 8 connection(s)
I20260619 12:11:01.042492 11985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:01.043272 12053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:01.046190 12053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe: Bootstrap starting.
I20260619 12:11:01.046828 12053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:01.047142 12053 log.cc:824] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:01.047870 12053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe: No bootstrap required, opened a new log
I20260619 12:11:01.050448 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 11985
I20260619 12:11:01.050549 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:11:01.050532 12053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3a509ce914d799e6ca83f11214abe" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 32935 } }
I20260619 12:11:01.050698 12053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:01.050716 804 external_mini_cluster.cc:1428] Setting key 50ce8e791a56a722179ba8205dfdd5c7
I20260619 12:11:01.050727 12053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb3a509ce914d799e6ca83f11214abe, State: Initialized, Role: FOLLOWER
I20260619 12:11:01.050859 12053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3a509ce914d799e6ca83f11214abe" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 32935 } }
I20260619 12:11:01.050917 12053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:01.050944 12053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:01.050979 12053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:01.051671 12053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3a509ce914d799e6ca83f11214abe" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 32935 } }
I20260619 12:11:01.051851 12053 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb3a509ce914d799e6ca83f11214abe; no voters:
I20260619 12:11:01.052006 12053 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:11:01.052049 12058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:01.052325 12058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [term 1 LEADER]: Becoming Leader. State: Replica: 8fb3a509ce914d799e6ca83f11214abe, State: Running, Role: LEADER
I20260619 12:11:01.052325 12053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:01.052512 12058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3a509ce914d799e6ca83f11214abe" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 32935 } }
I20260619 12:11:01.054575 12060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb3a509ce914d799e6ca83f11214abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3a509ce914d799e6ca83f11214abe" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 32935 } } }
I20260619 12:11:01.054647 12060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:01.054982 12059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb3a509ce914d799e6ca83f11214abe. Latest consensus state: current_term: 1 leader_uuid: "8fb3a509ce914d799e6ca83f11214abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb3a509ce914d799e6ca83f11214abe" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 32935 } } }
I20260619 12:11:01.055059 12059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:01.055337 12072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:01.055502 12053 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Jun 19 12:11:01 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:01.058526 12072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:01.063805 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:01.051298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.0.201.62:32935)
0619 12:11:01.051485 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:01.051492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:01.051515 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:11:01.051604 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:01.051609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:01.051649 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:01.051784 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:01.052563 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:01.053867 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:01.054942 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:01.055560 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:01.055670 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:01.058696 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:01.058745 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:01.058745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:01.058834 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:01.061716 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:01.062125 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:01.062127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:01.062128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:01.062231 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:01.062407 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:01.062408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:01.062408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:01.062786 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:01.062941 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:01.063438 (+ 497us) server_negotiation.cc:299] Negotiation successful
0619 12:11:01.063581 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260619 12:11:01.070075 12072 catalog_manager.cc:1347] Generated new cluster ID: 75e6fd016b1d4bf8b7a69e10da0fbc25
I20260619 12:11:01.070127 12072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:11:01.077255 12072 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:11:01.077996 12072 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:01.090128 12072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe: Generated new TSK 0
I20260619 12:11:01.090354 12072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:11:01.902561 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:56279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:32935
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:02.011907 12098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:02.012176 12098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:02.012216 12098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:02.013602 12098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:02.013674 12098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:02.013722 12098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:11:02.015515 12098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.201.20:56279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:32935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:02.015781 12098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:02.016034 12098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:02.021503 12103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:02.022508 12106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:02.022584 12098 server_base.cc:1034] running on GCE node
W20260619 12:11:02.022948 12104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:02.023178 12098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:02.023506 12098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:02.024668 12098 hybrid_clock.cc:648] HybridClock initialized: now 1781871062024658 us; error 29 us; skew 500 ppm
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:02.027446 12098 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:11:02.027958 12098 webserver.cc:460] Webserver started at http://127.0.201.1:32805/ using document root <none> and password file <none>
I20260619 12:11:02.028263 12098 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:02.028347 12098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:02.028493 12098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:02.041937 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "6db1bab06115408db7203113ebfab0d1"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "b976d38d87c46c760e5043a5cd194097"
server_key_iv: "c024e85b50ca5ee5bc264f6662d76697"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.042469 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "6db1bab06115408db7203113ebfab0d1"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "b976d38d87c46c760e5043a5cd194097"
server_key_iv: "c024e85b50ca5ee5bc264f6662d76697"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.047566 12098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260619 12:11:02.062366 12114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.062830 12098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260619 12:11:02.062983 12098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "6db1bab06115408db7203113ebfab0d1"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "b976d38d87c46c760e5043a5cd194097"
server_key_iv: "c024e85b50ca5ee5bc264f6662d76697"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.063141 12098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:02.094015 12098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:02.096299 12098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:02.096515 12098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:02.096799 12098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:02.097213 12098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:02.097304 12098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.097432 12098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:02.097513 12098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.103986 12098 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:36347
I20260619 12:11:02.104450 12098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:11:02.106988 12228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:36347 every 8 connection(s)
I20260619 12:11:02.109400 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 12098
I20260619 12:11:02.109498 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:11:02.109704 804 external_mini_cluster.cc:1428] Setting key 935cf9a7adee465c247a698fe7336abd
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:02.122095 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.107713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:32935 (local address 127.0.201.1:56597)
0619 12:11:02.108381 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.108431 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.109027 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.112121 (+ 3094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.112126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.112365 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:02.112649 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.112655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.113693 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.113694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.114960 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.114963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.115473 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.115474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.115626 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.116213 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:02.116224 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:02.117751 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:02.119955 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:02.119966 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:02.119968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:02.120292 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:02.120899 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:02.120900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:02.120901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:02.121078 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:02.121387 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:02.121393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:02.121573 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.121675 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.121844 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":109,"threads_started":1}
I20260619 12:11:02.122859 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.107932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:56597 (local address 127.0.201.62:32935)
0619 12:11:02.110389 (+ 2457us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.110390 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.110400 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:11:02.110421 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.110422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.111934 (+ 1512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.112740 (+ 806us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:02.112778 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.113588 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.115060 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.115368 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.115464 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.117865 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:02.117866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:02.117867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:02.117874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:02.119824 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.120388 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.120389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.120390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.120456 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.121165 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.121166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.121166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.121306 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:02.122584 (+ 1278us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.122633 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.122711 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2436,"thread_start_us":53,"threads_started":1}
I20260619 12:11:02.123464 12229 heartbeater.cc:347] Connected to a master server at 127.0.201.62:32935
I20260619 12:11:02.123556 12229 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
I20260619 12:11:02.123749 12229 heartbeater.cc:511] Master 127.0.201.62:32935 requested a full tablet report, sending...
I20260619 12:11:02.124300 12017 ts_manager.cc:194] Registered new tserver with Master: 6db1bab06115408db7203113ebfab0d1 (127.0.201.1:36347)
I20260619 12:11:02.125221 12017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:56597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:11:02.142416 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:56279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:32935
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:02.249259 12235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:02.249486 12235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:02.249526 12235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:02.250660 12235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:02.250754 12235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:02.250797 12235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:11:02.252592 12235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.201.20:56279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:32935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:02.252838 12235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:02.253091 12235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:02.258935 12245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:02.259308 12242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:02.259528 12235 server_base.cc:1034] running on GCE node
W20260619 12:11:02.260047 12243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:02.260385 12235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:02.260673 12235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:02.261826 12235 hybrid_clock.cc:648] HybridClock initialized: now 1781871062261802 us; error 38 us; skew 500 ppm
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:02.264899 12235 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:11:02.265389 12235 webserver.cc:460] Webserver started at http://127.0.201.2:39991/ using document root <none> and password file <none>
I20260619 12:11:02.265655 12235 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:02.265722 12235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:02.265854 12235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:02.278458 12235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "6dcbf75c3037466c93b7d0284c7db7f5"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "6e01c04e735e0cb7dffc9b801eb6bbb1"
server_key_iv: "9768f06f7f7d24d1e2a04abc9798c7d5"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.278842 12235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "6dcbf75c3037466c93b7d0284c7db7f5"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "6e01c04e735e0cb7dffc9b801eb6bbb1"
server_key_iv: "9768f06f7f7d24d1e2a04abc9798c7d5"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.280311 12235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:11:02.316753 12252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.317018 12235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260619 12:11:02.317118 12235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "6dcbf75c3037466c93b7d0284c7db7f5"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "6e01c04e735e0cb7dffc9b801eb6bbb1"
server_key_iv: "9768f06f7f7d24d1e2a04abc9798c7d5"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.317219 12235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:02.374545 12235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:02.380100 12235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:02.380316 12235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:02.380612 12235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:02.381039 12235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:02.381132 12235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.381203 12235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:02.381269 12235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.388823 12235 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:33979
I20260619 12:11:02.388976 12366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:33979 every 8 connection(s)
I20260619 12:11:02.389318 12235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:11:02.389801 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 12235
I20260619 12:11:02.389907 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:11:02.390129 804 external_mini_cluster.cc:1428] Setting key 442bea645974269df5d6b1aa349c919b
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:02.402776 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.390114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:32935 (local address 127.0.201.2:34735)
0619 12:11:02.390402 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.390446 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.391097 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.391391 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.391396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.391646 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:02.391997 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.392003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.392774 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.392775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.394078 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.394081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.394589 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.394590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.394736 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.395368 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:02.395378 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:02.397005 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:02.401138 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:02.401145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:02.401147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:02.401429 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:02.401646 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:02.401647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:02.401647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:02.401777 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:02.402125 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:02.402131 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:02.402302 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.402407 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.402553 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":101,"threads_started":1}
I20260619 12:11:02.403314 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.390120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:34735 (local address 127.0.201.62:32935)
0619 12:11:02.390172 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.390175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.390673 (+ 498us) server_negotiation.cc:407] Connection header received
0619 12:11:02.391222 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.391224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.391237 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.391310 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:02.392089 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.392698 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.394182 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.394504 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.394580 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.399287 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:02.399288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:02.399289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:02.399297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:02.401044 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.401520 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.401521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.401521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.401579 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.401886 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.401887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.401887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.402039 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:02.402099 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.403104 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.403218 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:11:02.404253 12367 heartbeater.cc:347] Connected to a master server at 127.0.201.62:32935
I20260619 12:11:02.404345 12367 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:02.404527 12367 heartbeater.cc:511] Master 127.0.201.62:32935 requested a full tablet report, sending...
I20260619 12:11:02.404874 12017 ts_manager.cc:194] Registered new tserver with Master: 6dcbf75c3037466c93b7d0284c7db7f5 (127.0.201.2:33979)
I20260619 12:11:02.405340 12017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:34735
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:11:02.430437 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:56279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:32935
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:02.560495 12372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:02.560833 12372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:02.560923 12372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:02.562649 12372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:02.562757 12372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:02.562834 12372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:11:02.565068 12372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.201.20:56279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:32935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:02.565495 12372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:02.565773 12372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:02.571895 12378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:02.572160 12380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:02.572321 12377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:02.573237 12372 server_base.cc:1034] running on GCE node
I20260619 12:11:02.573586 12372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:02.573822 12372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:02.574983 12372 hybrid_clock.cc:648] HybridClock initialized: now 1781871062574964 us; error 30 us; skew 500 ppm
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:02.578094 12372 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:11:02.579727 12372 webserver.cc:460] Webserver started at http://127.0.201.3:41631/ using document root <none> and password file <none>
I20260619 12:11:02.579969 12372 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:02.580015 12372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:02.580133 12372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:02.593735 12372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "3b01fa4caf3c49009a310d40fa27979e"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "bf31db28968d9f446946cb0855dc7f98"
server_key_iv: "8502fd517504873c6e609ba8c9e14557"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.594182 12372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "3b01fa4caf3c49009a310d40fa27979e"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "bf31db28968d9f446946cb0855dc7f98"
server_key_iv: "8502fd517504873c6e609ba8c9e14557"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.596076 12372 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:11:02.608505 12387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.608841 12372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:02.608999 12372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "3b01fa4caf3c49009a310d40fa27979e"
format_stamp: "Formatted at 2026-06-19 12:11:02 on dist-test-slave-c910"
server_key: "bf31db28968d9f446946cb0855dc7f98"
server_key_iv: "8502fd517504873c6e609ba8c9e14557"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:02.609161 12372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:02.629691 12372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:02.634684 12372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:02.634902 12372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:02.635234 12372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:02.635663 12372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:02.635754 12372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.635825 12372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:02.635881 12372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:02.654029 12372 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:44291
I20260619 12:11:02.654639 12372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:11:02.657797 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:44291 every 8 connection(s)
I20260619 12:11:02.664620 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 12372
I20260619 12:11:02.664726 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:11:02.664907 804 external_mini_cluster.cc:1428] Setting key 951bf102bca7b56e436ce1227ff655b2
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:02.676998 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.658531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:46563 (local address 127.0.201.62:32935)
0619 12:11:02.658590 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.658593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.659227 (+ 634us) server_negotiation.cc:407] Connection header received
0619 12:11:02.660406 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.660418 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.660432 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.660464 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:02.661626 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.662445 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.666484 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.666787 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.666831 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.672143 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:02.672144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:02.672145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:02.672152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:02.673979 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.675471 (+ 1492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.675472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.675473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.675533 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.676126 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.676128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.676128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.676277 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:02.676334 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.676818 (+ 484us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.676900 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 12:11:02.677745 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.658538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:32935 (local address 127.0.201.3:46563)
0619 12:11:02.659073 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.659113 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.660275 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.660655 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.660659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.660911 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:02.661513 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.661520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.663592 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.663593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.666362 (+ 2769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.666365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.666922 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.666923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.667061 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.669770 (+ 2709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:02.669780 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:02.672026 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:02.674639 (+ 2613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:02.674645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:02.674648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:02.674953 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:02.675626 (+ 673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:02.675626 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:02.675627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:02.675772 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:02.676441 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:02.676448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:02.676622 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.677333 (+ 711us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.677513 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":84,"threads_started":1}
I20260619 12:11:02.678627 12501 heartbeater.cc:347] Connected to a master server at 127.0.201.62:32935
I20260619 12:11:02.678717 12501 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:02.678896 12501 heartbeater.cc:511] Master 127.0.201.62:32935 requested a full tablet report, sending...
I20260619 12:11:02.679317 12017 ts_manager.cc:194] Registered new tserver with Master: 3b01fa4caf3c49009a310d40fa27979e (127.0.201.3:44291)
I20260619 12:11:02.679827 12017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:46563
I20260619 12:11:02.687333 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:11:02.702180 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.689982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.0.201.62:32935)
0619 12:11:02.690020 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.690022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.690312 (+ 290us) server_negotiation.cc:407] Connection header received
0619 12:11:02.690402 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.690403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.690425 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.690465 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:02.691168 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.691906 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.692978 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.693245 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.693289 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.695243 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:02.695245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:02.695245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:02.695252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:02.697831 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.700764 (+ 2933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.700765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.700765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.700825 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:02.701051 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:02.701052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:02.701052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:02.701179 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:02.701255 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.702015 (+ 760us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.702090 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:11:02.704392 12017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:11:02.706801 12017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:11:02.724742 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.717517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36347 (local address 127.0.0.1:35918)
0619 12:11:02.719365 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.719375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.719444 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.719725 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.719728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.719730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:02.719797 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.719800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.721541 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.721542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.722904 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.722907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.724406 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.724419 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.724506 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.724520 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.724598 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.724650 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1834,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260619 12:11:02.730562 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.717758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.0.201.1:36347)
0619 12:11:02.719313 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.719318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.719527 (+ 209us) server_negotiation.cc:407] Connection header received
0619 12:11:02.719565 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.719568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.719584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.719648 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:02.719880 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.721443 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.723008 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.724237 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.724311 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.724370 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.730434 (+ 6064us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.730481 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1521,"thread_start_us":49,"threads_started":1}
I20260619 12:11:02.731551 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.718009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:44291 (local address 127.0.0.1:41958)
0619 12:11:02.725170 (+ 7161us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.725179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.725212 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.725488 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.725489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.725491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:02.725520 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.725522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.728236 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.728237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.729344 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.729347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.731313 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.731314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.731375 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.731379 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.731444 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.731480 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7151,"thread_start_us":74,"threads_started":1}
I20260619 12:11:02.731721 12163 tablet_service.cc:1467] Processing CreateTablet for tablet fb6fdb9f9de643049d7c0bda7a181b98 (DEFAULT_TABLE table=test-table [id=9e29a1744056498198cc5ae0ba8a63cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:02.731966 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.718248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.0.201.3:44291)
0619 12:11:02.718646 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.718651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.725266 (+ 6615us) server_negotiation.cc:407] Connection header received
0619 12:11:02.725311 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.725313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.725328 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.725392 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:02.725596 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.728151 (+ 2555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.729471 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.730241 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.730308 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.730378 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.731788 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.731829 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":72,"threads_started":1}
I20260619 12:11:02.732132 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6fdb9f9de643049d7c0bda7a181b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:02.732882 12433 tablet_service.cc:1467] Processing CreateTablet for tablet fb6fdb9f9de643049d7c0bda7a181b98 (DEFAULT_TABLE table=test-table [id=9e29a1744056498198cc5ae0ba8a63cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:02.733209 12433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6fdb9f9de643049d7c0bda7a181b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:02.737761 12515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.717516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:33979 (local address 127.0.0.1:34904)
0619 12:11:02.718710 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.718754 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.720859 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.723255 (+ 2396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.723256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.723258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:02.723297 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.723298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.726930 (+ 3632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.726931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.728056 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.728059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.737508 (+ 9449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.737509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.737581 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.737586 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.737636 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.737675 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"thread_start_us":71,"threads_started":1}
I20260619 12:11:02.737819 12516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.717615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34904 (local address 127.0.201.2:33979)
0619 12:11:02.723026 (+ 5411us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.723030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.723040 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:11:02.723076 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.723078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.723093 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.723162 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:02.725617 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.726663 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.732321 (+ 5658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.733067 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.733139 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.733208 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.737672 (+ 4464us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.737732 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5379,"thread_start_us":70,"threads_started":1}
I20260619 12:11:02.738431 12521 tablet_bootstrap.cc:492] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Bootstrap starting.
I20260619 12:11:02.738646 12301 tablet_service.cc:1467] Processing CreateTablet for tablet fb6fdb9f9de643049d7c0bda7a181b98 (DEFAULT_TABLE table=test-table [id=9e29a1744056498198cc5ae0ba8a63cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:02.738996 12301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6fdb9f9de643049d7c0bda7a181b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:02.739514 12521 tablet_bootstrap.cc:654] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:02.739794 12521 log.cc:824] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:02.740525 12521 tablet_bootstrap.cc:492] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: No bootstrap required, opened a new log
I20260619 12:11:02.740583 12521 ts_tablet_manager.cc:1397] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:11:02.742548 12521 raft_consensus.cc:348] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.742700 12521 raft_consensus.cc:374] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:02.742728 12521 raft_consensus.cc:729] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b01fa4caf3c49009a310d40fa27979e, State: Initialized, Role: FOLLOWER
I20260619 12:11:02.742825 12521 consensus_queue.cc:260] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.743033 12521 ts_tablet_manager.cc:1428] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:11:02.743323 12501 heartbeater.cc:503] Master 127.0.201.62:32935 was elected leader, sending a full tablet report...
I20260619 12:11:02.744948 12523 tablet_bootstrap.cc:492] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: Bootstrap starting.
I20260619 12:11:02.745994 12523 tablet_bootstrap.cc:654] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:02.746352 12523 log.cc:824] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:02.747079 12523 tablet_bootstrap.cc:492] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: No bootstrap required, opened a new log
I20260619 12:11:02.747136 12523 ts_tablet_manager.cc:1397] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:11:02.749433 12523 raft_consensus.cc:348] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.749591 12523 raft_consensus.cc:374] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:02.749621 12523 raft_consensus.cc:729] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcbf75c3037466c93b7d0284c7db7f5, State: Initialized, Role: FOLLOWER
I20260619 12:11:02.749720 12523 consensus_queue.cc:260] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.749930 12523 ts_tablet_manager.cc:1428] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:11:02.750031 12367 heartbeater.cc:503] Master 127.0.201.62:32935 was elected leader, sending a full tablet report...
I20260619 12:11:02.750339 12520 tablet_bootstrap.cc:492] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: Bootstrap starting.
I20260619 12:11:02.751062 12520 tablet_bootstrap.cc:654] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:02.751384 12520 log.cc:824] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:02.755770 12520 tablet_bootstrap.cc:492] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: No bootstrap required, opened a new log
I20260619 12:11:02.755892 12520 ts_tablet_manager.cc:1397] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260619 12:11:02.758234 12520 raft_consensus.cc:348] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.758459 12520 raft_consensus.cc:374] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:02.758534 12520 raft_consensus.cc:729] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6db1bab06115408db7203113ebfab0d1, State: Initialized, Role: FOLLOWER
I20260619 12:11:02.758713 12520 consensus_queue.cc:260] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.759011 12520 ts_tablet_manager.cc:1428] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 12:11:02.759176 12229 heartbeater.cc:503] Master 127.0.201.62:32935 was elected leader, sending a full tablet report...
Jun 19 12:11:02 dist-test-slave-c910 krb5kdc[11541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:02.828783 12524 raft_consensus.cc:482] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:11:02.828933 12524 raft_consensus.cc:504] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.829350 12524 leader_election.cc:288] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6db1bab06115408db7203113ebfab0d1 (127.0.201.1:36347), 6dcbf75c3037466c93b7d0284c7db7f5 (127.0.201.2:33979)
I20260619 12:11:02.833575 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.829574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36347 (local address 127.0.201.3:37327)
0619 12:11:02.829931 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.829943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.829983 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.830280 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.830281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.830283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:02.830352 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.830354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.831306 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.831307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.832512 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.832515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.833306 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.833307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.833379 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.833383 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.833444 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.833482 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340}
I20260619 12:11:02.833957 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.830078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:37327 (local address 127.0.201.1:36347)
0619 12:11:02.830121 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.830124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.830132 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:11:02.830164 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.830165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.830175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.830225 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:02.830445 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.831163 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.832614 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.833196 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.833265 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.833275 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.833827 (+ 552us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.833875 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:11:02.834229 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6fdb9f9de643049d7c0bda7a181b98" candidate_uuid: "3b01fa4caf3c49009a310d40fa27979e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6db1bab06115408db7203113ebfab0d1" is_pre_election: true
I20260619 12:11:02.834394 12183 raft_consensus.cc:2454] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b01fa4caf3c49009a310d40fa27979e in term 0.
I20260619 12:11:02.834628 12389 leader_election.cc:302] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b01fa4caf3c49009a310d40fa27979e, 6db1bab06115408db7203113ebfab0d1; no voters:
I20260619 12:11:02.834744 12524 raft_consensus.cc:2781] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:11:02.834779 12524 raft_consensus.cc:482] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:11:02.834797 12524 raft_consensus.cc:3037] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:02.835721 12524 raft_consensus.cc:504] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.835850 12524 leader_election.cc:288] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [CANDIDATE]: Term 1 election: Requested vote from peers 6db1bab06115408db7203113ebfab0d1 (127.0.201.1:36347), 6dcbf75c3037466c93b7d0284c7db7f5 (127.0.201.2:33979)
I20260619 12:11:02.836020 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6fdb9f9de643049d7c0bda7a181b98" candidate_uuid: "3b01fa4caf3c49009a310d40fa27979e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6db1bab06115408db7203113ebfab0d1"
I20260619 12:11:02.836079 12183 raft_consensus.cc:3037] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:02.836942 12183 raft_consensus.cc:2454] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b01fa4caf3c49009a310d40fa27979e in term 1.
I20260619 12:11:02.837098 12389 leader_election.cc:302] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b01fa4caf3c49009a310d40fa27979e, 6db1bab06115408db7203113ebfab0d1; no voters:
I20260619 12:11:02.837164 12524 raft_consensus.cc:2781] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:02.837296 12524 raft_consensus.cc:686] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 1 LEADER]: Becoming Leader. State: Replica: 3b01fa4caf3c49009a310d40fa27979e, State: Running, Role: LEADER
I20260619 12:11:02.837379 12524 consensus_queue.cc:237] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } }
I20260619 12:11:02.837791 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.829721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:33979 (local address 127.0.201.3:60155)
0619 12:11:02.833651 (+ 3930us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.833663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:02.833710 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:02.833998 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:02.833999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:02.834000 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:02.834033 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.834035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.835169 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.835171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.836220 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:02.836223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.837560 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.837561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:02.837630 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.837633 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:11:02.837681 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:11:02.837716 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3918,"thread_start_us":119,"threads_started":1}
I20260619 12:11:02.838322 12016 catalog_manager.cc:5495] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b01fa4caf3c49009a310d40fa27979e (127.0.201.3). New cstate: current_term: 1 leader_uuid: "3b01fa4caf3c49009a310d40fa27979e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b01fa4caf3c49009a310d40fa27979e" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 } health_report { overall_health: UNKNOWN } } }
I20260619 12:11:02.839507 12516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.829767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:60155 (local address 127.0.201.2:33979)
0619 12:11:02.829813 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.829822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.833846 (+ 4024us) server_negotiation.cc:407] Connection header received
0619 12:11:02.833875 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.833876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.833890 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.833951 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:02.834249 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.834922 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.836749 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.837343 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.837427 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.837438 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.839400 (+ 1962us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.839442 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:11:02.839763 12318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6fdb9f9de643049d7c0bda7a181b98" candidate_uuid: "3b01fa4caf3c49009a310d40fa27979e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" is_pre_election: true
I20260619 12:11:02.839890 12318 raft_consensus.cc:2454] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b01fa4caf3c49009a310d40fa27979e in term 0.
I20260619 12:11:02.839996 12319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb6fdb9f9de643049d7c0bda7a181b98" candidate_uuid: "3b01fa4caf3c49009a310d40fa27979e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcbf75c3037466c93b7d0284c7db7f5"
I20260619 12:11:02.840039 12319 raft_consensus.cc:3037] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:02.840871 12319 raft_consensus.cc:2454] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b01fa4caf3c49009a310d40fa27979e in term 1.
W20260619 12:11:02.857977 12230 tablet.cc:2369] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:11:02.871228 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.867746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.0.201.3:44291)
0619 12:11:02.867816 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:02.867818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:02.867846 (+ 28us) server_negotiation.cc:407] Connection header received
0619 12:11:02.867913 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:02.867914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:02.867935 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:02.868001 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:11:02.868210 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.868968 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.870157 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:02.870470 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:02.870528 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:02.870718 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:11:02.870843 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:11:02.870932 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:02.871094 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 12:11:02.871114 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:11:02.873620 12183 raft_consensus.cc:1261] T fb6fdb9f9de643049d7c0bda7a181b98 P 6db1bab06115408db7203113ebfab0d1 [term 1 FOLLOWER]: Refusing update from remote peer 3b01fa4caf3c49009a310d40fa27979e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:02.873620 12319 raft_consensus.cc:1261] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 1 FOLLOWER]: Refusing update from remote peer 3b01fa4caf3c49009a310d40fa27979e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:02.873975 12530 consensus_queue.cc:1035] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dcbf75c3037466c93b7d0284c7db7f5" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:02.874209 12530 consensus_queue.cc:1035] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6db1bab06115408db7203113ebfab0d1" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:02.878211 12533 mvcc.cc:205] Tried to move back new op lower bound from 7298543873527324672 to 7298543873382617088. Current Snapshot: MvccSnapshot[applied={T|T < 7298543873527324672}]
I20260619 12:11:02.878583 12534 mvcc.cc:205] Tried to move back new op lower bound from 7298543873527324672 to 7298543873382617088. Current Snapshot: MvccSnapshot[applied={T|T < 7298543873527324672}]
I20260619 12:11:02.879429 12535 mvcc.cc:205] Tried to move back new op lower bound from 7298543873527324672 to 7298543873382617088. Current Snapshot: MvccSnapshot[applied={T|T < 7298543873527324672}]
I20260619 12:11:02.881369 12017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:02.881454 12017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:02.882241 12017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fb3a509ce914d799e6ca83f11214abe: Sending DeleteTablet for 3 replicas of tablet fb6fdb9f9de643049d7c0bda7a181b98
I20260619 12:11:02.882550 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet fb6fdb9f9de643049d7c0bda7a181b98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:02 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:35918
I20260619 12:11:02.882958 12301 tablet_service.cc:1514] Processing DeleteTablet for tablet fb6fdb9f9de643049d7c0bda7a181b98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:02 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:34904
I20260619 12:11:02.883323 12433 tablet_service.cc:1514] Processing DeleteTablet for tablet fb6fdb9f9de643049d7c0bda7a181b98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:02 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:41958
I20260619 12:11:02.884397 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 12098
W20260619 12:11:02.894667 12002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:36347 (error 108)
I20260619 12:11:02.894726 12543 tablet_replica.cc:330] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: stopping tablet replica
I20260619 12:11:02.894824 12543 raft_consensus.cc:2229] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:11:02.894930 12543 raft_consensus.cc:2258] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:02.895092 12544 tablet_replica.cc:330] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: stopping tablet replica
I20260619 12:11:02.895184 12544 raft_consensus.cc:2229] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:11:02.895309 12544 raft_consensus.cc:2258] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:02.895749 12543 ts_tablet_manager.cc:1905] T fb6fdb9f9de643049d7c0bda7a181b98 P 6dcbf75c3037466c93b7d0284c7db7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:02.895810 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 12235
I20260619 12:11:02.895829 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.895699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36347 (local address 127.0.0.1:35928)
0619 12:11:02.895746 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.895792 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 12:11:02.896749 12544 ts_tablet_manager.cc:1905] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:02.898252 12544 ts_tablet_manager.cc:1918] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:02.898309 12544 log.cc:1196] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/fb6fdb9f9de643049d7c0bda7a181b98
I20260619 12:11:02.898561 12544 ts_tablet_manager.cc:1939] T fb6fdb9f9de643049d7c0bda7a181b98 P 3b01fa4caf3c49009a310d40fa27979e: Deleting consensus metadata
I20260619 12:11:02.899679 12004 catalog_manager.cc:4841] TS 3b01fa4caf3c49009a310d40fa27979e (127.0.201.3:44291): tablet fb6fdb9f9de643049d7c0bda7a181b98 (table test-table [id=9e29a1744056498198cc5ae0ba8a63cd]) successfully deleted
W20260619 12:11:02.899933 12002 catalog_manager.cc:4584] TS 6db1bab06115408db7203113ebfab0d1 (127.0.201.1:36347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb6fdb9f9de643049d7c0bda7a181b98: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36347: connect: Connection refused (error 111)
I20260619 12:11:02.902858 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 12372
I20260619 12:11:02.903957 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:02.903840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:33979 (local address 127.0.0.1:34906)
0619 12:11:02.903898 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:02.903928 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.2:33979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:11:02.908593 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 11985
I20260619 12:11:02.914403 804 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 12:11:03.260780 804 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 12:11:03.260893 804 mini_ranger.cc:66] Stopping Ranger...
I20260619 12:11:03.604895 804 mini_ranger.cc:68] Stopped Ranger
2026-06-19 12:11:03.605 UTC [11571] LOG: received smart shutdown request
2026-06-19 12:11:03.608 UTC [11571] LOG: background worker "logical replication launcher" (PID 11580) exited with exit code 1
2026-06-19 12:11:03.608 UTC [11575] LOG: shutting down
2026-06-19 12:11:03.750 UTC [11571] LOG: database system is shut down
2026-06-19T12:11:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44510 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:11:03 dist-test-slave-c910 krb5kdc[12554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:11:03 dist-test-slave-c910 krb5kdc[12554](info): set up 2 sockets
Jun 19 12:11:03 dist-test-slave-c910 krb5kdc[12554](info): commencing operation
krb5kdc: starting...
W20260619 12:11:04.895159 804 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:11:04 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:11:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/rangeradmin_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/rangeradmin_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/rangerlookup_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/rangerlookup_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
I20260619 12:11:05.045756 804 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq9ki8M/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/postgres -l logfile start
2026-06-19 12:11:07.590 UTC [12584] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 12:11:07.590 UTC [12584] LOG: listening on IPv4 address "127.0.201.20", port 60049
2026-06-19 12:11:07.594 UTC [12584] LOG: listening on Unix socket "/tmp/.s.PGSQL.60049"
2026-06-19 12:11:07.628 UTC [12587] LOG: database system was shut down at 2026-06-19 12:11:06 UTC
2026-06-19 12:11:07.632 UTC [12584] LOG: database system is ready to accept connections
I20260619 12:11:08.575393 804 mini_postgres.cc:96] Postgres bound to 60049
2026-06-19 12:11:08.581 UTC [12595] FATAL: database "slave" does not exist
127.0.201.20:60049 - accepting connections
I20260619 12:11:08.581898 804 mini_ranger.cc:161] Starting Ranger...
I20260619 12:11:08.596231 804 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 12:11:09.080783 804 mini_ranger.cc:87] Created ranger Postgres database
I20260619 12:11:09.080863 804 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-admin
2026-06-19 12:11:09,352 [I] DB FLAVOR :POSTGRES
2026-06-19 12:11:09,352 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:11:09,353 [I] Checking connection..
2026-06-19 12:11:09,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 12:11:09,584 [I] Checking connection passed.
2026-06-19 12:11:09,585 [I] --------- Verifying version history table ---------
2026-06-19 12:11:09,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:11:09,846 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:11:09,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:11:10,101 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:11:10,101 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 12:11:10,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 12:11:10.362 UTC [12659] WARNING: there is no transaction in progress
2026-06-19 12:11:10,368 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 12:11:10,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:11:10,626 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 12:11:10,626 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:11:10,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 12:11:10,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 12:11:11,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c910.c.gcp-upstream.internal','N') ;"
2026-06-19 12:11:11,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:11:11,581 [I] Table x_portal_user does not exist in database ranger
2026-06-19 12:11:11,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:11:11,833 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 12:11:11,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:11:12,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:11:12,283 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 12:11:12,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 12:11:13.204 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.219 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.235 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.250 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.264 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.281 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.334 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.341 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.355 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.365 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.375 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.389 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.398 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.404 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.411 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:13.418 UTC [12767] WARNING: there is no transaction in progress
2026-06-19 12:11:14,007 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 12:11:14,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 12:11:14,223 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 12:11:14,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:11:14,475 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 12:11:14,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:11:14,714 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 12:11:14,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:11:14,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:11:15,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c910.c.gcp-upstream.internal';"
2026-06-19 12:11:15,456 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 12:11:15,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 12:11:15,741 [I] DB_PATCHES have already been applied
I20260619 12:11:15.746411 804 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq9ki8M/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 12:11:15.746488 804 mini_ranger.cc:193] Using host: 127.0.201.20
I20260619 12:11:15.749672 804 mini_ranger.cc:227] Ranger admin URL: http://127.0.201.20:55269
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 12:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 12:11:16.844581 804 mini_ranger.cc:160] Time spent starting Ranger: real 8.263s user 0.001s sys 0.007s
Jun 19, 2026 12:11:36 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 12:11:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 12:11:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 12:11:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 12:11:37 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 12:11:38.102676 804 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.201.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.201.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/rangerkms_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.201.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/rangerkms_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.201.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.201.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.201.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/HTTP_127.0.201.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 12:11:38 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871098, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 12:11:38.185334 804 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 12:11:38.194027 804 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 12:11:39.084342 804 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 12:11:39.084415 804 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms
I20260619 12:11:39.084430 804 mini_ranger_kms.cc:227] Using postgres at 127.0.201.20:60049
2026-06-19 12:11:39,249 [I] DB FLAVOR :POSTGRES
2026-06-19 12:11:39,249 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:11:39,249 [I] Checking connection
2026-06-19 12:11:39,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 12:11:39,500 [I] connection success
2026-06-19 12:11:39,500 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 12:11:39,500 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 12:11:39,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 12:11:39,759 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 12:11:39,759 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:11:39,759 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 12:11:39,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.201.20:60049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 12:11:40,043 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 12:11:40.497102 804 mini_ranger_kms.cc:310] Created kms service
I20260619 12:11:40.683633 804 mini_ranger_kms.cc:326] Created kudu user
I20260619 12:11:40.725457 804 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 12:11:40 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871098, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:41.111900 804 mini_ranger_kms.cc:384] Added ranger policy
I20260619 12:11:41.112015 804 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq9ki8M/build/release/bin/postgresql.jar:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq9ki8M/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 12:11:41.112035 804 mini_ranger_kms.cc:240] Using host: 127.0.201.20
I20260619 12:11:41.115355 804 mini_ranger_kms.cc:276] Ranger KMS PID: 12947
I20260619 12:11:41.115432 804 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.201.20:39229
Jun 19, 2026 12:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 12:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 12:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 12:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 12:11:43 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.201.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:43.367583 804 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.182s user 0.008s sys 0.140s
I20260619 12:11:43.367702 804 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 12:11:43.367753 804 mini_ranger_kms.cc:401] 127.0.201.20:39229/kms/v1/keys
Jun 19 12:11:43 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.201.20@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
Jun 19 12:11:44 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.201.20@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
Jun 19 12:11:46 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:11:46.118397 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:33651
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:33651
--ranger_config_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:46.196709 12998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:46.196943 12998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:46.196978 12998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:46.198089 12998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:46.198161 12998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:46.198184 12998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:46.198205 12998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:46.198236 12998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:46.199786 12998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.201.20:39229/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:33651
--ranger_config_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:33651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:46.200060 12998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:46.200276 12998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:46.205331 13006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:46.205332 13003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:46.205364 13004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:46.205359 12998 server_base.cc:1034] running on GCE node
I20260619 12:11:46.205778 12998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:46.206004 12998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:46.207137 12998 hybrid_clock.cc:648] HybridClock initialized: now 1781871106207127 us; error 29 us; skew 500 ppm
Jun 19 12:11:46 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:46.209595 12998 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:46.210026 12998 webserver.cc:460] Webserver started at http://127.0.201.62:39097/ using document root <none> and password file <none>
I20260619 12:11:46.210248 12998 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:46.210306 12998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:46.210459 12998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:46 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:46.262573 12998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "554536353c7946e484e30f87af9b74fd"
format_stamp: "Formatted at 2026-06-19 12:11:46 on dist-test-slave-c910"
server_key: "fa5a1d2a3d5b3bb2ddb8fdf832d8e367"
server_key_iv: "d95ad3862f53cd71e1bff41cacbc58e6"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:46.262898 12998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "554536353c7946e484e30f87af9b74fd"
format_stamp: "Formatted at 2026-06-19 12:11:46 on dist-test-slave-c910"
server_key: "fa5a1d2a3d5b3bb2ddb8fdf832d8e367"
server_key_iv: "d95ad3862f53cd71e1bff41cacbc58e6"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:46.264523 12998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:11:46.275118 13013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:46.275350 12998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:11:46.275460 12998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "554536353c7946e484e30f87af9b74fd"
format_stamp: "Formatted at 2026-06-19 12:11:46 on dist-test-slave-c910"
server_key: "fa5a1d2a3d5b3bb2ddb8fdf832d8e367"
server_key_iv: "d95ad3862f53cd71e1bff41cacbc58e6"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:46.275564 12998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:46.283928 12998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:46.285894 12998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:46.286049 12998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:46.290750 12998 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:33651
I20260619 12:11:46.290781 13065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:33651 every 8 connection(s)
I20260619 12:11:46.291167 12998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:46.291838 13066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:46.293529 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 12998
I20260619 12:11:46.293625 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:11:46.293838 804 external_mini_cluster.cc:1428] Setting key d070370017711198f792d7d218f2c94d
I20260619 12:11:46.294217 13066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Bootstrap starting.
I20260619 12:11:46.294889 13066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:46.295233 13066 log.cc:824] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:46.295964 13066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: No bootstrap required, opened a new log
I20260619 12:11:46.297936 13066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:46.298098 13066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:46.298173 13066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 554536353c7946e484e30f87af9b74fd, State: Initialized, Role: FOLLOWER
I20260619 12:11:46.298362 13066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:46.298462 13066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:46.298508 13066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:46.298588 13066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:46.299361 13066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:46.299489 13066 leader_election.cc:302] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 554536353c7946e484e30f87af9b74fd; no voters:
I20260619 12:11:46.299656 13066 leader_election.cc:288] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:11:46.299719 13071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:46.299867 13071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 LEADER]: Becoming Leader. State: Replica: 554536353c7946e484e30f87af9b74fd, State: Running, Role: LEADER
I20260619 12:11:46.299937 13066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:46.299978 13071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
Jun 19 12:11:46 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:46.301338 13072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "554536353c7946e484e30f87af9b74fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } } }
I20260619 12:11:46.301430 13072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:46.301575 13072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 554536353c7946e484e30f87af9b74fd. Latest consensus state: current_term: 1 leader_uuid: "554536353c7946e484e30f87af9b74fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } } }
I20260619 12:11:46.301635 13072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:46.301759 13082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:46.301983 13066 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:11:46.302325 13082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:46.307554 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:46.294398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.0.201.62:33651)
0619 12:11:46.294593 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:46.294601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:46.294628 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:11:46.295152 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:46.295161 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:46.295262 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:46.295436 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:46.296122 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:46.296985 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:46.297761 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:46.298214 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:46.298366 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:46.301296 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:46.301300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:46.301301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:46.301312 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:46.303027 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:46.303430 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:46.303432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:46.303433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:46.305196 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:46.305493 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:46.305499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:46.305499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:46.306289 (+ 790us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:46.306648 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:46.306740 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 12:11:46.307208 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260619 12:11:46.311703 13082 catalog_manager.cc:1347] Generated new cluster ID: 1070be1dd80a43c0ba77a1dd2c54ecdd
I20260619 12:11:46.311762 13082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:11:46.327446 13082 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:11:46.328244 13082 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:46.338791 13082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Generated new TSK 0
I20260619 12:11:46.339000 13082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:11:46.998787 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33651
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:47.118420 13111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:47.118641 13111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:47.118680 13111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:47.119997 13111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:47.120061 13111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:47.120107 13111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:11:47.121671 13111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.201.20:39229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:47.121924 13111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:47.122145 13111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:47.127084 13119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.126986 13117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.126996 13116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:47.127528 13111 server_base.cc:1034] running on GCE node
I20260619 12:11:47.127719 13111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:47.127933 13111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:47.129081 13111 hybrid_clock.cc:648] HybridClock initialized: now 1781871107129052 us; error 44 us; skew 500 ppm
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:47.131946 13111 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:11:47.132517 13111 webserver.cc:460] Webserver started at http://127.0.201.1:36579/ using document root <none> and password file <none>
I20260619 12:11:47.132797 13111 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:47.132864 13111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:47.133000 13111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:47.145115 13111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "deb578f6aa674702a791470d6e8929e0"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "de5b707ad5907b5d03a1b2aa93473a0b"
server_key_iv: "80f436401decaeae837a9cfb9d4d5f57"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.145584 13111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "deb578f6aa674702a791470d6e8929e0"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "de5b707ad5907b5d03a1b2aa93473a0b"
server_key_iv: "80f436401decaeae837a9cfb9d4d5f57"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.147414 13111 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:11:47.160912 13126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.161092 13111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:47.161176 13111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "deb578f6aa674702a791470d6e8929e0"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "de5b707ad5907b5d03a1b2aa93473a0b"
server_key_iv: "80f436401decaeae837a9cfb9d4d5f57"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.161267 13111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:47.186534 13111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:47.189050 13111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:47.189188 13111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:47.189395 13111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:47.189744 13111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:47.189793 13111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.189823 13111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:47.189841 13111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.196897 13111 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:38177
I20260619 12:11:47.197019 13241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:38177 every 8 connection(s)
I20260619 12:11:47.197365 13111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:11:47.198513 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13111
I20260619 12:11:47.198617 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:11:47.198836 804 external_mini_cluster.cc:1428] Setting key f4715a50ffba5177298b9880b96d1021
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
I20260619 12:11:47.209453 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:47.198146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33651 (local address 127.0.201.1:36789)
0619 12:11:47.198305 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:47.198347 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:47.198974 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:47.199330 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:47.199335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:47.199616 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:47.199949 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:47.199955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.201043 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.201044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:47.202444 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:47.202447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.203023 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.203024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:47.203165 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:47.203782 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:47.203792 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:47.205552 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:47.207759 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:47.207765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:47.207767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:47.208072 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:47.208302 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:47.208304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:47.208304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:47.208452 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:47.208747 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:47.208752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:47.208929 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 12:11:47.209053 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 12:11:47.209211 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260619 12:11:47.209815 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:47.198388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:36789 (local address 127.0.201.62:33651)
0619 12:11:47.198526 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:47.198528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:47.198538 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:11:47.199133 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:47.199134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:47.199148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:47.199251 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:47.200050 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.200938 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.202536 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.202869 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.202933 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:47.205665 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:47.205667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:47.205667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:47.205675 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:47.207669 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:47.208164 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:47.208165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:47.208166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:47.208226 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:47.208533 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:47.208534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:47.208534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:47.208667 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:47.208722 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:47.209623 (+ 901us) server_negotiation.cc:299] Negotiation successful
0619 12:11:47.209708 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260619 12:11:47.210525 13242 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33651
I20260619 12:11:47.210623 13242 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:47.210804 13242 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:47.211496 13030 ts_manager.cc:194] Registered new tserver with Master: deb578f6aa674702a791470d6e8929e0 (127.0.201.1:38177)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
I20260619 12:11:47.212373 13030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:36789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:11:47.229730 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33651
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:47.335130 13248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:47.335389 13248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:47.335445 13248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:47.336608 13248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:47.336674 13248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:47.336719 13248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:11:47.338157 13248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.201.20:39229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:47.338398 13248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:47.338642 13248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:47.344075 13258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.344125 13256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.344367 13255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:47.344933 13248 server_base.cc:1034] running on GCE node
I20260619 12:11:47.345167 13248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:47.345424 13248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:47.346582 13248 hybrid_clock.cc:648] HybridClock initialized: now 1781871107346563 us; error 33 us; skew 500 ppm
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:47.349432 13248 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:11:47.349850 13248 webserver.cc:460] Webserver started at http://127.0.201.2:45625/ using document root <none> and password file <none>
I20260619 12:11:47.350085 13248 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:47.350152 13248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:47.350281 13248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:47.364722 13248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "c5b3ea4b367445799131c219a6d469a3"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "b989b3116a3305d0fd01802b9c006b84"
server_key_iv: "14d6300bf3ccf5a64e56bc782b46ebdc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.365257 13248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "c5b3ea4b367445799131c219a6d469a3"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "b989b3116a3305d0fd01802b9c006b84"
server_key_iv: "14d6300bf3ccf5a64e56bc782b46ebdc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.369676 13248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 12:11:47.391604 13265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.391881 13248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:11:47.391973 13248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "c5b3ea4b367445799131c219a6d469a3"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "b989b3116a3305d0fd01802b9c006b84"
server_key_iv: "14d6300bf3ccf5a64e56bc782b46ebdc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.392068 13248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:47.412974 13248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:47.415385 13248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:47.415563 13248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:47.415827 13248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:47.416231 13248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:47.416288 13248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.416335 13248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:47.416364 13248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.424024 13248 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:34593
I20260619 12:11:47.424479 13248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:11:47.425283 13378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:34593 every 8 connection(s)
I20260619 12:11:47.430461 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13248
I20260619 12:11:47.430588 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:11:47.430814 804 external_mini_cluster.cc:1428] Setting key 93a3993b40192ffad72baa01b62a41ae
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:47.437119 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:47.426170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:40953 (local address 127.0.201.62:33651)
0619 12:11:47.426217 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:47.426219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:47.426432 (+ 213us) server_negotiation.cc:407] Connection header received
0619 12:11:47.426955 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:47.426957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:47.426969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:47.427015 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:47.427687 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.428258 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.429468 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.429684 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.429763 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:47.434122 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:47.434123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:47.434123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:47.434132 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:47.435649 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:47.436098 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:47.436099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:47.436100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:47.436161 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:47.436413 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:47.436414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:47.436414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:47.436577 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:47.436639 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:47.436938 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 12:11:47.437029 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:11:47.437371 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:47.425844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33651 (local address 127.0.201.2:40953)
0619 12:11:47.426288 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:47.426333 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:47.426852 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:47.427094 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:47.427098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:47.427380 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:47.427588 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:47.427593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.428347 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.428348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:47.429369 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:47.429372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.429779 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.429780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:47.429897 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:47.431099 (+ 1202us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:47.431111 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:47.434026 (+ 2915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:47.435786 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:47.435792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:47.435794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:47.436019 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:47.436227 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:47.436228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:47.436228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:47.436345 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:47.436642 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:47.436648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:47.436799 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 12:11:47.436904 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:11:47.437059 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":77,"threads_started":1}
I20260619 12:11:47.437870 13379 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33651
I20260619 12:11:47.437995 13379 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:47.438179 13379 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:47.438580 13030 ts_manager.cc:194] Registered new tserver with Master: c5b3ea4b367445799131c219a6d469a3 (127.0.201.2:34593)
I20260619 12:11:47.438951 13030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:40953
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:11:47.461575 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33651
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:47.551620 13385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:47.551844 13385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:47.551887 13385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:47.553186 13385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:47.553256 13385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:47.553303 13385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:11:47.555352 13385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.201.20:39229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:47.555626 13385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:47.555848 13385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:47.561156 13390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.561270 13391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.561551 13393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:47.561806 13385 server_base.cc:1034] running on GCE node
I20260619 12:11:47.562044 13385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:47.562314 13385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:47.563458 13385 hybrid_clock.cc:648] HybridClock initialized: now 1781871107563438 us; error 41 us; skew 500 ppm
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:47.565999 13385 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:11:47.566499 13385 webserver.cc:460] Webserver started at http://127.0.201.3:35983/ using document root <none> and password file <none>
I20260619 12:11:47.566833 13385 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:47.566903 13385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:47.567042 13385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:47.580518 13385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "7560343409924f88876c9e4235b48d7a"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "66a0c06293c6cf6f44507339f673c3d0"
server_key_iv: "2262c9ed0a77102829e1aa826cb4b17d"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.581032 13385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "7560343409924f88876c9e4235b48d7a"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "66a0c06293c6cf6f44507339f673c3d0"
server_key_iv: "2262c9ed0a77102829e1aa826cb4b17d"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.583367 13385 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:11:47.598017 13400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.598270 13385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:47.598364 13385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "7560343409924f88876c9e4235b48d7a"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "66a0c06293c6cf6f44507339f673c3d0"
server_key_iv: "2262c9ed0a77102829e1aa826cb4b17d"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.598454 13385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:47.632421 13385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:47.634522 13385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:47.634680 13385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:47.634907 13385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:47.635383 13385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:47.635442 13385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.635478 13385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:47.635509 13385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.643796 13385 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:36017
I20260619 12:11:47.643991 13513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:36017 every 8 connection(s)
I20260619 12:11:47.644240 13385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:11:47.648902 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13385
I20260619 12:11:47.648999 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:11:47.649210 804 external_mini_cluster.cc:1428] Setting key 4c8aea48b9ece5456e7a5913dc59e9fa
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:47.663137 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:47.645213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:47807 (local address 127.0.201.62:33651)
0619 12:11:47.645255 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:47.645257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:47.652504 (+ 7247us) server_negotiation.cc:407] Connection header received
0619 12:11:47.653218 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:47.653219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:47.653231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:47.653263 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:47.654119 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.654859 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.656411 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.656718 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.656764 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:47.659367 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:47.659368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:47.659369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:47.659376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:47.661179 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:47.662009 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:47.662010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:47.662011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:47.662078 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:47.662355 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:47.662356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:47.662356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:47.662507 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:47.662558 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:47.662944 (+ 386us) server_negotiation.cc:299] Negotiation successful
0619 12:11:47.663026 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:11:47.663646 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:47.644937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33651 (local address 127.0.201.3:47807)
0619 12:11:47.652363 (+ 7426us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:47.652416 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:47.653071 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:47.653457 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:47.653462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:47.653733 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:47.654009 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:47.654015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.654983 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.654984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:47.656296 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:47.656299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:47.656866 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:47.656867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:47.657003 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:47.657650 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:47.657660 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:47.659261 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:47.661615 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:47.661622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:47.661624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:47.661912 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:47.662145 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:47.662146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:47.662146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:47.662271 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:47.662591 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:47.662596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:47.662763 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 12:11:47.663257 (+ 494us) client_negotiation.cc:240] Negotiation successful
0619 12:11:47.663416 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7373,"thread_start_us":83,"threads_started":1}
I20260619 12:11:47.664089 13514 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33651
I20260619 12:11:47.664186 13514 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:47.664368 13514 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:47.664685 13030 ts_manager.cc:194] Registered new tserver with Master: 7560343409924f88876c9e4235b48d7a (127.0.201.3:36017)
I20260619 12:11:47.665184 13030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:47807
I20260619 12:11:47.672531 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:11:47.673357 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13111
I20260619 12:11:47.679418 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13248
I20260619 12:11:47.685732 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13385
I20260619 12:11:47.689949 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 12998
I20260619 12:11:47.698143 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:33651
--webserver_interface=127.0.201.62
--webserver_port=39097
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:33651
--ranger_config_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:47.779446 13517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:47.779631 13517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:47.779652 13517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:47.780702 13517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:47.780735 13517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:47.780750 13517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:47.780761 13517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:47.780772 13517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:47.782230 13517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:33651
--ranger_config_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:33651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:47.782403 13517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:47.782555 13517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:47.787230 13525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.787252 13523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.787235 13522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:47.787427 13517 server_base.cc:1034] running on GCE node
I20260619 12:11:47.787657 13517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:47.787942 13517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:47.789101 13517 hybrid_clock.cc:648] HybridClock initialized: now 1781871107789088 us; error 45 us; skew 500 ppm
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:47.791428 13517 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:47.791846 13517 webserver.cc:460] Webserver started at http://127.0.201.62:39097/ using document root <none> and password file <none>
I20260619 12:11:47.792027 13517 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:47.792074 13517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:47.831269 13517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:47.832057 13533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:47.832263 13517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:47.832373 13517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "554536353c7946e484e30f87af9b74fd"
format_stamp: "Formatted at 2026-06-19 12:11:46 on dist-test-slave-c910"
server_key: "fa5a1d2a3d5b3bb2ddb8fdf832d8e367"
server_key_iv: "d95ad3862f53cd71e1bff41cacbc58e6"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:47.832760 13517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:47.851994 13517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:47.853929 13517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:47.854091 13517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:47.858220 13517 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:33651
I20260619 12:11:47.858260 13585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:33651 every 8 connection(s)
I20260619 12:11:47.858618 13517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:47.860137 13586 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:11:47.860920 13586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Bootstrap starting.
I20260619 12:11:47.863929 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13517
I20260619 12:11:47.864216 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:38177
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:33651
--webserver_port=36579
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:47.868786 13586 log.cc:824] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:47.870404 13586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:11:47.870682 13586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Bootstrap complete.
I20260619 12:11:47.873052 13586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:47.873317 13586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 554536353c7946e484e30f87af9b74fd, State: Initialized, Role: FOLLOWER
I20260619 12:11:47.873525 13586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:47.873618 13586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:47.873672 13586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:47.873718 13586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:11:47.874676 13586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:47.874799 13586 leader_election.cc:302] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 554536353c7946e484e30f87af9b74fd; no voters:
I20260619 12:11:47.874976 13586 leader_election.cc:288] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:11:47.875038 13590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:11:47.875173 13590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [term 2 LEADER]: Becoming Leader. State: Replica: 554536353c7946e484e30f87af9b74fd, State: Running, Role: LEADER
I20260619 12:11:47.875257 13586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:47.875278 13590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } }
I20260619 12:11:47.875538 13590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 554536353c7946e484e30f87af9b74fd. Latest consensus state: current_term: 2 leader_uuid: "554536353c7946e484e30f87af9b74fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } } }
I20260619 12:11:47.875629 13590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:47.875962 13595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:47.875540 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "554536353c7946e484e30f87af9b74fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "554536353c7946e484e30f87af9b74fd" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 33651 } } }
I20260619 12:11:47.876142 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:47.876718 13595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:47.876839 13595 catalog_manager.cc:1259] Loaded cluster ID: 1070be1dd80a43c0ba77a1dd2c54ecdd
I20260619 12:11:47.876864 13595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:11:47.877228 13586 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:11:47.878151 13595 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:47.878310 13595 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Loaded TSK: 0
I20260619 12:11:47.878511 13595 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:11:47.957669 13588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:47.957895 13588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:47.957935 13588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:47.959256 13588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:47.959331 13588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:47.959374 13588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:11:47.960835 13588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:38177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=36579
--tserver_master_addrs=127.0.201.62:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:47.961079 13588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:47.961282 13588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:47.966516 13626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.966516 13624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:47.966800 13623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:47.967640 13588 server_base.cc:1034] running on GCE node
I20260619 12:11:47.967932 13588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:47.968179 13588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:47.975512 13588 hybrid_clock.cc:648] HybridClock initialized: now 1781871107975494 us; error 29 us; skew 500 ppm
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:47.978684 13588 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:11:47.979239 13588 webserver.cc:460] Webserver started at http://127.0.201.1:36579/ using document root <none> and password file <none>
I20260619 12:11:47.979507 13588 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:47.979585 13588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:11:47 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:48.002506 13588 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 12:11:48.003664 13634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.003834 13588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:48.003935 13588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "deb578f6aa674702a791470d6e8929e0"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "de5b707ad5907b5d03a1b2aa93473a0b"
server_key_iv: "80f436401decaeae837a9cfb9d4d5f57"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:48.004417 13588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:48.017859 13588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:48.020187 13588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:48.020362 13588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:48.020565 13588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:48.020929 13588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:48.020977 13588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.021006 13588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:48.021030 13588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.028292 13588 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:38177
I20260619 12:11:48.028725 13588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:11:48.030725 13747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:38177 every 8 connection(s)
I20260619 12:11:48.031497 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13588
I20260619 12:11:48.031807 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:34593
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:33651
--webserver_port=45625
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:48.045583 13750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.029935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33651 (local address 127.0.201.1:43287)
0619 12:11:48.030890 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:48.030942 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:48.031740 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:48.032383 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:48.032387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:48.032615 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:48.032863 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:48.032868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.034502 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.034503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:48.035817 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:48.035819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.036750 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.036752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:48.036882 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.037473 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:48.037483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:48.039082 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:48.043677 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:48.043683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:48.043685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:48.043974 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:48.044293 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:48.044294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:48.044294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:48.044426 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:48.044938 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:48.044944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:48.045110 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 12:11:48.045203 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 12:11:48.045354 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":82,"threads_started":1}
I20260619 12:11:48.046276 13751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.030284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:43287 (local address 127.0.201.62:33651)
0619 12:11:48.031310 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:48.031318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:48.031356 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:11:48.032042 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:48.032049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:48.032103 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:48.032228 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:48.033003 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.034378 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.035926 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.036513 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.036644 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.041440 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:48.041444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:48.041445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:48.041457 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:48.043568 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.044091 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.044094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.044095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.044210 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.044533 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.044534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.044534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.044844 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:48.045871 (+ 1027us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:48.045939 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:11:48.046066 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":67,"threads_started":1}
I20260619 12:11:48.046984 13748 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33651
I20260619 12:11:48.047091 13748 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.047303 13748 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
W20260619 12:11:48.048130 13748 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:33651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:11:48.048203 13748 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.048266 13748 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:48.048560 13748 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.048638 13748 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
W20260619 12:11:48.048910 13748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:11:48.147600 13752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:48.147812 13752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:48.147847 13752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:48.148984 13752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:48.149051 13752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:48.149097 13752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:11:48.151300 13752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:34593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=45625
--tserver_master_addrs=127.0.201.62:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:48.151718 13752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:48.151970 13752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:48.156916 13758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:48.156971 13760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:48.157658 13757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:48.157737 13752 server_base.cc:1034] running on GCE node
I20260619 12:11:48.158039 13752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:48.158319 13752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:48.159474 13752 hybrid_clock.cc:648] HybridClock initialized: now 1781871108159458 us; error 35 us; skew 500 ppm
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:48.162693 13752 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:11:48.163290 13752 webserver.cc:460] Webserver started at http://127.0.201.2:45625/ using document root <none> and password file <none>
I20260619 12:11:48.163601 13752 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:48.163750 13752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:48.180465 13752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:48.181332 13768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.181511 13752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:11:48.181600 13752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "c5b3ea4b367445799131c219a6d469a3"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "b989b3116a3305d0fd01802b9c006b84"
server_key_iv: "14d6300bf3ccf5a64e56bc782b46ebdc"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:48.181974 13752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:48.192613 13752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:48.196350 13752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:48.196497 13752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:48.196717 13752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:48.197089 13752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:48.197135 13752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.197163 13752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:48.197223 13752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.204149 13752 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:34593
I20260619 12:11:48.204571 13752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:11:48.205166 13881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:34593 every 8 connection(s)
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:48.213827 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13752
I20260619 12:11:48.214128 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.201.20:39229/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:36017
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:33651
--webserver_port=35983
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:48.215401 13751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.205580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:50145 (local address 127.0.201.62:33651)
0619 12:11:48.205631 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:48.205633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:48.205769 (+ 136us) server_negotiation.cc:407] Connection header received
0619 12:11:48.206463 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:48.206464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:48.206478 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:48.206512 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:48.207344 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.208110 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.209752 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.209979 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.210076 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.212377 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:48.212378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:48.212378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:48.212385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:48.213816 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.214307 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.214308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.214309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.214367 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.214648 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.214649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.214649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.214782 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:48.214823 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:48.215189 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 12:11:48.215304 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:11:48.215880 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.205276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33651 (local address 127.0.201.2:50145)
0619 12:11:48.205589 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:48.205686 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:48.206322 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:48.206707 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:48.206712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:48.206926 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:48.207227 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:48.207234 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.208281 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.208281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:48.209660 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:48.209663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.210107 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.210109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:48.210222 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.210707 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:48.210715 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:48.212273 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:48.213913 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:48.213918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:48.213921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:48.214205 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:48.214441 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:48.214442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:48.214442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:48.214568 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:48.214869 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:48.214875 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:48.215038 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 12:11:48.215515 (+ 477us) client_negotiation.cc:240] Negotiation successful
0619 12:11:48.215663 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":83,"threads_started":1}
I20260619 12:11:48.216527 13882 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33651
I20260619 12:11:48.216614 13882 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.216784 13882 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
W20260619 12:11:48.217391 13882 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:33651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:11:48.217453 13882 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.217509 13882 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:48.217797 13882 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.217860 13882 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
W20260619 12:11:48.218132 13882 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:11:48.323505 13885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:48.323721 13885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:48.323756 13885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:48.324966 13885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:48.325038 13885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:48.325083 13885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:11:48.326555 13885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.201.20:39229/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:36017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=35983
--tserver_master_addrs=127.0.201.62:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:48.326807 13885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:48.327054 13885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:48.331935 13891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:48.331995 13893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:48.332159 13890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:48.332721 13885 server_base.cc:1034] running on GCE node
I20260619 12:11:48.332962 13885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:48.333237 13885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:48.334414 13885 hybrid_clock.cc:648] HybridClock initialized: now 1781871108334385 us; error 46 us; skew 500 ppm
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:48.337098 13885 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:11:48.337595 13885 webserver.cc:460] Webserver started at http://127.0.201.3:35983/ using document root <none> and password file <none>
I20260619 12:11:48.337842 13885 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:48.337925 13885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:48.353291 13885 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 12:11:48.354319 13901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.354525 13885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:11:48.354595 13885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "7560343409924f88876c9e4235b48d7a"
format_stamp: "Formatted at 2026-06-19 12:11:47 on dist-test-slave-c910"
server_key: "66a0c06293c6cf6f44507339f673c3d0"
server_key_iv: "2262c9ed0a77102829e1aa826cb4b17d"
server_key_version: "kuduclusterkey@0"
I20260619 12:11:48.355037 13885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:48.365167 13885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:48.369138 13885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:48.369298 13885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:48.369513 13885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:48.369891 13885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:48.370007 13885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.370059 13885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:48.370079 13885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:48.377485 13885 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:36017
I20260619 12:11:48.378247 13885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:11:48.380492 14015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:36017 every 8 connection(s)
I20260619 12:11:48.385578 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 13885
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:48.391481 13751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.381097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:46531 (local address 127.0.201.62:33651)
0619 12:11:48.381180 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:48.381183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:48.381460 (+ 277us) server_negotiation.cc:407] Connection header received
0619 12:11:48.381996 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:48.381997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:48.382010 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:48.382072 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:48.382662 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.383328 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.384503 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.384739 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.384835 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.387972 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:48.387973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:48.387974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:48.387981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:48.389759 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.390232 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.390233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.390233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.390302 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.390638 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.390639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.390640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.390791 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:48.390860 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:48.391296 (+ 436us) server_negotiation.cc:299] Negotiation successful
0619 12:11:48.391386 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:11:48.392045 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.381091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:33651 (local address 127.0.201.3:46531)
0619 12:11:48.381335 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:48.381366 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:48.381882 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:48.382158 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:48.382161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:48.382338 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:48.382540 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:48.382544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.383447 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.383448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:48.384386 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:48.384389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.384864 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.384866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:48.385001 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.385905 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:48.385916 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:48.387846 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:48.389854 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:48.389860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:48.389862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:48.390141 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:48.390391 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:48.390392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:48.390392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:48.390543 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:48.390915 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:48.390921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:48.391088 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 12:11:48.391640 (+ 552us) client_negotiation.cc:240] Negotiation successful
0619 12:11:48.391806 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260619 12:11:48.392530 14016 heartbeater.cc:347] Connected to a master server at 127.0.201.62:33651
I20260619 12:11:48.392657 14016 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.392932 14016 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
W20260619 12:11:48.393657 14016 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:33651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:11:48.393764 14016 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.393865 14016 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:48.394196 14016 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:48.394281 14016 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
W20260619 12:11:48.394584 14016 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:11:48.396770 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:48.386149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.0.201.62:33651)
0619 12:11:48.387489 (+ 1340us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:48.387490 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:48.387496 (+ 6us) server_negotiation.cc:407] Connection header received
0619 12:11:48.387512 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:48.387512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:48.387520 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:48.387604 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:48.388437 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.389228 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.390378 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:48.390684 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:48.390730 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:48.392713 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:48.392714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:48.392715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:48.392722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:48.395402 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.395881 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.395882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.395883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.395947 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:48.396172 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:48.396173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:48.396174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:48.396290 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:48.396498 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:48.396546 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 12:11:48.396632 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":46,"threads_started":1}
Jun 19 12:11:48 dist-test-slave-c910 krb5kdc[12554](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:49.049108 13748 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:49.049256 13748 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:49.049758 13550 ts_manager.cc:194] Registered new tserver with Master: deb578f6aa674702a791470d6e8929e0 (127.0.201.1:38177)
I20260619 12:11:49.050781 13550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:43287
I20260619 12:11:49.218317 13882 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:49.218485 13882 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:49.218789 13550 ts_manager.cc:194] Registered new tserver with Master: c5b3ea4b367445799131c219a6d469a3 (127.0.201.2:34593)
I20260619 12:11:49.219399 13550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:50145
Jun 19 12:11:49 dist-test-slave-c910 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for HTTP/127.0.201.20@KRBTEST.COM
I20260619 12:11:49.394791 14016 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:49.394946 14016 heartbeater.cc:511] Master 127.0.201.62:33651 requested a full tablet report, sending...
I20260619 12:11:49.395326 13550 ts_manager.cc:194] Registered new tserver with Master: 7560343409924f88876c9e4235b48d7a (127.0.201.3:36017)
I20260619 12:11:49.395812 13550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:46531
I20260619 12:11:49.404043 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:11:49.411811 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.404927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.0.201.62:33651)
0619 12:11:49.405172 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.405174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.405179 (+ 5us) server_negotiation.cc:407] Connection header received
0619 12:11:49.405197 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.405198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.405205 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.405275 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:49.405887 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.406377 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.407411 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.407677 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.407751 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.409058 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:49.409059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:49.409060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:49.409066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:49.410625 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:49.410971 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:49.410973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:49.410973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:49.411030 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:49.411278 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:49.411279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:49.411279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:49.411392 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:49.411509 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.411605 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.411686 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":80,"threads_started":1}
I20260619 12:11:49.413879 13550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:11:49.415629 13550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:11:49.423969 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.418277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38177 (local address 127.0.0.1:37940)
0619 12:11:49.418696 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.418736 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:49.418801 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:49.419481 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:49.419485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:49.419498 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:49.419571 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.419573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.421090 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.421090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.422362 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.422365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.423696 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.423697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.423765 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.423778 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 12:11:49.423842 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:11:49.423888 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":47,"thread_start_us":85,"threads_started":1}
I20260619 12:11:49.424540 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.418542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37940 (local address 127.0.201.1:38177)
0619 12:11:49.419244 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.419248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.419258 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:11:49.419303 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.419305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.419319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.419392 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:49.419653 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.420749 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.422464 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.423550 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.423607 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.423666 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.424374 (+ 708us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.424415 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":53,"threads_started":1}
I20260619 12:11:49.424911 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.418277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:34593 (local address 127.0.0.1:36666)
0619 12:11:49.418950 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.418959 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:49.419022 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:49.419491 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:49.419491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:49.419493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:49.419922 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.419923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.421074 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.421075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.423488 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.423490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.424666 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.424667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.424754 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.424759 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:11:49.424809 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:11:49.424845 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":1604,"thread_start_us":80,"threads_started":1}
I20260619 12:11:49.425235 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.418711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.0.201.2:34593)
0619 12:11:49.418901 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.418906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.419080 (+ 174us) server_negotiation.cc:407] Connection header received
0619 12:11:49.419120 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.419122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.419139 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.419235 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:49.420003 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.420981 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.423579 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.424336 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.425046 (+ 710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.425112 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.425137 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.425180 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":64,"threads_started":1}
I20260619 12:11:49.425369 13682 tablet_service.cc:1467] Processing CreateTablet for tablet b6acbb427f104fff9e8a0992ea8b4ece (DEFAULT_TABLE table=test-table [id=21d513582db2466ba5ab8a6d940fce1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:49.425743 13682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6acbb427f104fff9e8a0992ea8b4ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:49.425868 13816 tablet_service.cc:1467] Processing CreateTablet for tablet b6acbb427f104fff9e8a0992ea8b4ece (DEFAULT_TABLE table=test-table [id=21d513582db2466ba5ab8a6d940fce1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:49.426163 13816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6acbb427f104fff9e8a0992ea8b4ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:49.427726 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.418515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:36017 (local address 127.0.0.1:44954)
0619 12:11:49.419383 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.419393 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:49.419433 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:49.419802 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:49.419803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:49.419805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:49.419835 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.419836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.425547 (+ 5711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.425548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.426582 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.426584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.427527 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.427528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.427589 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.427594 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:11:49.427627 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 12:11:49.427661 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":68,"threads_started":1}
I20260619 12:11:49.428093 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.418615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.0.201.3:36017)
0619 12:11:49.419597 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.419602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.419611 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:11:49.419645 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.419647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.419661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.419729 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:49.421308 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.422455 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.426685 (+ 4230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.427421 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.427878 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.427936 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.427963 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.428019 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":61,"threads_started":1}
I20260619 12:11:49.428642 13948 tablet_service.cc:1467] Processing CreateTablet for tablet b6acbb427f104fff9e8a0992ea8b4ece (DEFAULT_TABLE table=test-table [id=21d513582db2466ba5ab8a6d940fce1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:49.428916 13948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6acbb427f104fff9e8a0992ea8b4ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:49.429953 14050 tablet_bootstrap.cc:492] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: Bootstrap starting.
I20260619 12:11:49.430644 14050 tablet_bootstrap.cc:654] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:49.430997 14050 log.cc:824] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:49.431763 14050 tablet_bootstrap.cc:492] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: No bootstrap required, opened a new log
I20260619 12:11:49.431829 14050 ts_tablet_manager.cc:1397] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 12:11:49.432575 14052 tablet_bootstrap.cc:492] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Bootstrap starting.
I20260619 12:11:49.433167 14052 tablet_bootstrap.cc:654] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:49.433439 14052 log.cc:824] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:49.434114 14052 tablet_bootstrap.cc:492] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: No bootstrap required, opened a new log
I20260619 12:11:49.434170 14052 ts_tablet_manager.cc:1397] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:11:49.434267 14050 raft_consensus.cc:348] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.434415 14050 raft_consensus.cc:374] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:49.434451 14050 raft_consensus.cc:729] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5b3ea4b367445799131c219a6d469a3, State: Initialized, Role: FOLLOWER
I20260619 12:11:49.435097 14050 consensus_queue.cc:260] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.435483 14050 ts_tablet_manager.cc:1428] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260619 12:11:49.435606 14049 tablet_bootstrap.cc:492] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: Bootstrap starting.
I20260619 12:11:49.435644 13882 heartbeater.cc:503] Master 127.0.201.62:33651 was elected leader, sending a full tablet report...
I20260619 12:11:49.436246 14049 tablet_bootstrap.cc:654] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:49.436621 14049 log.cc:824] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:49.436774 14052 raft_consensus.cc:348] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.436964 14052 raft_consensus.cc:374] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:49.437021 14052 raft_consensus.cc:729] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7560343409924f88876c9e4235b48d7a, State: Initialized, Role: FOLLOWER
I20260619 12:11:49.437141 14052 consensus_queue.cc:260] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.437502 14016 heartbeater.cc:503] Master 127.0.201.62:33651 was elected leader, sending a full tablet report...
I20260619 12:11:49.437493 14052 ts_tablet_manager.cc:1428] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 12:11:49.443754 14049 tablet_bootstrap.cc:492] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: No bootstrap required, opened a new log
I20260619 12:11:49.443835 14049 ts_tablet_manager.cc:1397] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 12:11:49.445729 14049 raft_consensus.cc:348] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.445873 14049 raft_consensus.cc:374] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:49.445935 14049 raft_consensus.cc:729] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deb578f6aa674702a791470d6e8929e0, State: Initialized, Role: FOLLOWER
I20260619 12:11:49.446055 14049 consensus_queue.cc:260] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.446326 14049 ts_tablet_manager.cc:1428] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:11:49.446336 13748 heartbeater.cc:503] Master 127.0.201.62:33651 was elected leader, sending a full tablet report...
W20260619 12:11:49.456081 13883 tablet.cc:2369] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:11:49.531678 13749 tablet.cc:2369] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:11:49.537881 14056 raft_consensus.cc:482] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:11:49.538008 14056 raft_consensus.cc:504] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.538388 14056 leader_election.cc:288] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers deb578f6aa674702a791470d6e8929e0 (127.0.201.1:38177), c5b3ea4b367445799131c219a6d469a3 (127.0.201.2:34593)
I20260619 12:11:49.542474 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.538836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:34557 (local address 127.0.201.1:38177)
0619 12:11:49.538888 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.538890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.538953 (+ 63us) server_negotiation.cc:407] Connection header received
0619 12:11:49.539054 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.539056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.539068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.539097 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:49.539337 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.540093 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.541593 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.542107 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.542154 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.542165 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.542366 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.542409 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:11:49.542536 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.538946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:38261 (local address 127.0.201.2:34593)
0619 12:11:49.538986 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.538987 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.539130 (+ 143us) server_negotiation.cc:407] Connection header received
0619 12:11:49.539263 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.539264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.539277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.539306 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:49.539532 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.540227 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.541603 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.542135 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.542182 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.542192 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.542426 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.542467 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:11:49.542830 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.538726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:34593 (local address 127.0.201.3:38261)
0619 12:11:49.539037 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.539048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:49.539160 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:49.539373 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:49.539374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:49.539375 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:49.539449 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.539450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.540329 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.540330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.541485 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.541488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.542252 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.542253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.542341 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.542344 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:11:49.542709 (+ 365us) client_negotiation.cc:240] Negotiation successful
0619 12:11:49.542751 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260619 12:11:49.542954 14058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.538570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38177 (local address 127.0.201.3:34557)
0619 12:11:49.538889 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.538900 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:49.538981 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:49.539162 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:49.539163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:49.539165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:49.539262 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.539264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.540190 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.540191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.541491 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:49.541494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.542224 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.542225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:49.542289 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.542292 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:11:49.542817 (+ 525us) client_negotiation.cc:240] Negotiation successful
0619 12:11:49.542859 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260619 12:11:49.543385 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acbb427f104fff9e8a0992ea8b4ece" candidate_uuid: "7560343409924f88876c9e4235b48d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5b3ea4b367445799131c219a6d469a3" is_pre_election: true
I20260619 12:11:49.543551 13836 raft_consensus.cc:2454] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7560343409924f88876c9e4235b48d7a in term 0.
I20260619 12:11:49.543782 13906 leader_election.cc:302] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7560343409924f88876c9e4235b48d7a, c5b3ea4b367445799131c219a6d469a3; no voters:
I20260619 12:11:49.543972 14056 raft_consensus.cc:2781] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:11:49.544044 14056 raft_consensus.cc:482] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:11:49.544066 14056 raft_consensus.cc:3037] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:49.544915 14056 raft_consensus.cc:504] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.545038 14056 leader_election.cc:288] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [CANDIDATE]: Term 1 election: Requested vote from peers deb578f6aa674702a791470d6e8929e0 (127.0.201.1:38177), c5b3ea4b367445799131c219a6d469a3 (127.0.201.2:34593)
I20260619 12:11:49.545176 13683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acbb427f104fff9e8a0992ea8b4ece" candidate_uuid: "7560343409924f88876c9e4235b48d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deb578f6aa674702a791470d6e8929e0" is_pre_election: true
I20260619 12:11:49.545301 13683 raft_consensus.cc:2454] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7560343409924f88876c9e4235b48d7a in term 0.
I20260619 12:11:49.545552 13683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acbb427f104fff9e8a0992ea8b4ece" candidate_uuid: "7560343409924f88876c9e4235b48d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deb578f6aa674702a791470d6e8929e0"
I20260619 12:11:49.545636 13683 raft_consensus.cc:3037] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:49.546420 13683 raft_consensus.cc:2454] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7560343409924f88876c9e4235b48d7a in term 1.
I20260619 12:11:49.546532 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acbb427f104fff9e8a0992ea8b4ece" candidate_uuid: "7560343409924f88876c9e4235b48d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5b3ea4b367445799131c219a6d469a3"
I20260619 12:11:49.546594 13836 raft_consensus.cc:3037] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:49.546633 13904 leader_election.cc:302] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7560343409924f88876c9e4235b48d7a, deb578f6aa674702a791470d6e8929e0; no voters:
I20260619 12:11:49.547365 14056 raft_consensus.cc:2781] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:49.547392 13836 raft_consensus.cc:2454] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7560343409924f88876c9e4235b48d7a in term 1.
I20260619 12:11:49.547514 14056 raft_consensus.cc:686] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 1 LEADER]: Becoming Leader. State: Replica: 7560343409924f88876c9e4235b48d7a, State: Running, Role: LEADER
I20260619 12:11:49.547617 14056 consensus_queue.cc:237] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } }
I20260619 12:11:49.548390 13550 catalog_manager.cc:5495] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7560343409924f88876c9e4235b48d7a (127.0.201.3). New cstate: current_term: 1 leader_uuid: "7560343409924f88876c9e4235b48d7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7560343409924f88876c9e4235b48d7a" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 36017 } health_report { overall_health: HEALTHY } } }
I20260619 12:11:49.572427 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.569158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.0.201.3:36017)
0619 12:11:49.569278 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:49.569280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:49.569289 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:11:49.569328 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:49.569329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:49.569353 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:49.569430 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:11:49.569623 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.570337 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.571476 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:49.571782 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:49.571861 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:49.572052 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:11:49.572189 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:11:49.572279 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:49.572326 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 12:11:49.572348 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 12:11:49.574582 13836 raft_consensus.cc:1261] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 1 FOLLOWER]: Refusing update from remote peer 7560343409924f88876c9e4235b48d7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:49.574641 13683 raft_consensus.cc:1261] T b6acbb427f104fff9e8a0992ea8b4ece P deb578f6aa674702a791470d6e8929e0 [term 1 FOLLOWER]: Refusing update from remote peer 7560343409924f88876c9e4235b48d7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:49.575008 14056 consensus_queue.cc:1035] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5b3ea4b367445799131c219a6d469a3" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 34593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:49.575122 14060 consensus_queue.cc:1035] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "deb578f6aa674702a791470d6e8929e0" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 38177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:49.577702 14063 mvcc.cc:205] Tried to move back new op lower bound from 7298544064814723072 to 7298544064707629056. Current Snapshot: MvccSnapshot[applied={T|T < 7298544064814723072}]
I20260619 12:11:49.579619 14065 mvcc.cc:205] Tried to move back new op lower bound from 7298544064814723072 to 7298544064707629056. Current Snapshot: MvccSnapshot[applied={T|T < 7298544064814723072}]
I20260619 12:11:49.581449 13549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:49.581562 13549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:49.582336 13549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 554536353c7946e484e30f87af9b74fd: Sending DeleteTablet for 3 replicas of tablet b6acbb427f104fff9e8a0992ea8b4ece
I20260619 12:11:49.582808 13682 tablet_service.cc:1514] Processing DeleteTablet for tablet b6acbb427f104fff9e8a0992ea8b4ece with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:49 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:37940
I20260619 12:11:49.583024 13816 tablet_service.cc:1514] Processing DeleteTablet for tablet b6acbb427f104fff9e8a0992ea8b4ece with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:49 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36666
I20260619 12:11:49.583439 13948 tablet_service.cc:1514] Processing DeleteTablet for tablet b6acbb427f104fff9e8a0992ea8b4ece with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:49 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:44954
I20260619 12:11:49.583988 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13588
I20260619 12:11:49.585633 14073 tablet_replica.cc:330] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3: stopping tablet replica
I20260619 12:11:49.585891 14073 raft_consensus.cc:2229] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:11:49.586651 14073 raft_consensus.cc:2258] T b6acbb427f104fff9e8a0992ea8b4ece P c5b3ea4b367445799131c219a6d469a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:49.587651 14074 tablet_replica.cc:330] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: stopping tablet replica
I20260619 12:11:49.587800 14074 raft_consensus.cc:2229] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:11:49.587960 14074 raft_consensus.cc:2258] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:11:49.589881 13535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:38177 (error 108)
I20260619 12:11:49.590755 14074 ts_tablet_manager.cc:1905] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:49.590827 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:49.590691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:38177 (local address 127.0.0.1:37942)
0619 12:11:49.590739 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.590773 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:38177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:11:49.591003 13535 catalog_manager.cc:4584] TS deb578f6aa674702a791470d6e8929e0 (127.0.201.1:38177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6acbb427f104fff9e8a0992ea8b4ece: Network error: Client connection negotiation failed: client connection to 127.0.201.1:38177: connect: Connection refused (error 111)
I20260619 12:11:49.591274 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13752
I20260619 12:11:49.593844 14074 ts_tablet_manager.cc:1918] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:49.596611 14074 log.cc:1196] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/b6acbb427f104fff9e8a0992ea8b4ece
I20260619 12:11:49.597923 14074 ts_tablet_manager.cc:1939] T b6acbb427f104fff9e8a0992ea8b4ece P 7560343409924f88876c9e4235b48d7a: Deleting consensus metadata
W20260619 12:11:49.597990 14045 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:11:49.597632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:34593 (local address 127.0.0.1:36680)
0619 12:11:49.597681 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:49.597692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:49.597734 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:49.597940 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.2:34593: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 12:11:49.598438 13534 catalog_manager.cc:4841] TS 7560343409924f88876c9e4235b48d7a (127.0.201.3:36017): tablet b6acbb427f104fff9e8a0992ea8b4ece (table test-table [id=21d513582db2466ba5ab8a6d940fce1e]) successfully deleted
I20260619 12:11:49.598721 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13885
I20260619 12:11:49.605665 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 13517
I20260619 12:11:49.612183 804 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 12:11:49.946411 804 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 12:11:49.946496 804 mini_ranger.cc:66] Stopping Ranger...
I20260619 12:11:50.291052 804 mini_ranger.cc:68] Stopped Ranger
2026-06-19 12:11:50.291 UTC [12584] LOG: received smart shutdown request
2026-06-19 12:11:50.295 UTC [12584] LOG: background worker "logical replication launcher" (PID 12593) exited with exit code 1
2026-06-19 12:11:50.295 UTC [12588] LOG: shutting down
2026-06-19 12:11:50.464 UTC [12584] LOG: database system is shut down
2026-06-19T12:11:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46716 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:11:50 dist-test-slave-c910 krb5kdc[14084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:11:50 dist-test-slave-c910 krb5kdc[14084](info): set up 2 sockets
Jun 19 12:11:50 dist-test-slave-c910 krb5kdc[14084](info): commencing operation
krb5kdc: starting...
W20260619 12:11:51.603281 804 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:11:51 dist-test-slave-c910 krb5kdc[14084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:11:51.725556 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:41351
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:41351,127.0.201.61:32805,127.0.201.60:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:51.805243 14097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:51.805441 14097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:51.805464 14097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:51.806510 14097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:51.806545 14097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:51.806558 14097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:51.806571 14097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:51.806583 14097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:51.807955 14097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:41351,127.0.201.61:32805,127.0.201.60:41817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:41351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:51.808131 14097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:51.808286 14097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:11:51.813045 14097 server_base.cc:1034] running on GCE node
W20260619 12:11:51.812980 14103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:51.812985 14105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:51.812987 14102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:51.813627 14097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:51.813880 14097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:51.815034 14097 hybrid_clock.cc:648] HybridClock initialized: now 1781871111815015 us; error 43 us; skew 500 ppm
Jun 19 12:11:51 dist-test-slave-c910 krb5kdc[14084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:51.822495 14097 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:51.822904 14097 webserver.cc:460] Webserver started at http://127.0.201.62:41415/ using document root <none> and password file <none>
I20260619 12:11:51.823098 14097 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:51.823139 14097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:51.823297 14097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:51.824292 14097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "2db40b5b15c0431487330af8fd178ce3"
format_stamp: "Formatted at 2026-06-19 12:11:51 on dist-test-slave-c910"
server_key: "c30899397b66c726b5bce41c530049e9"
server_key_iv: "491add80ed39d84f8734c004a9fbcb91"
server_key_version: "encryptionkey@0"
I20260619 12:11:51.824580 14097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "2db40b5b15c0431487330af8fd178ce3"
format_stamp: "Formatted at 2026-06-19 12:11:51 on dist-test-slave-c910"
server_key: "c30899397b66c726b5bce41c530049e9"
server_key_iv: "491add80ed39d84f8734c004a9fbcb91"
server_key_version: "encryptionkey@0"
I20260619 12:11:51.825774 14097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:11:51.826442 14112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:51.826612 14097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:51.826696 14097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "2db40b5b15c0431487330af8fd178ce3"
format_stamp: "Formatted at 2026-06-19 12:11:51 on dist-test-slave-c910"
server_key: "c30899397b66c726b5bce41c530049e9"
server_key_iv: "491add80ed39d84f8734c004a9fbcb91"
server_key_version: "encryptionkey@0"
I20260619 12:11:51.826748 14097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:51.849836 14097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:51.852154 14097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:51.852263 14097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:51.857103 14097 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:41351
I20260619 12:11:51.857179 14164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:41351 every 8 connection(s)
I20260619 12:11:51.857429 14097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:51.858203 14165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:51.859756 14165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:51.860105 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14097
I20260619 12:11:51.860201 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:11:51.860396 804 external_mini_cluster.cc:1428] Setting key e922b313514ced0c9f96ce36792a63c3
Jun 19 12:11:51 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.61@KRBTEST.COM; defaulting to no policy
I20260619 12:11:51.869387 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.860516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.0.201.62:41351)
0619 12:11:51.860719 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:51.860724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:51.860771 (+ 47us) server_negotiation.cc:407] Connection header received
0619 12:11:51.861281 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:51.861284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:51.861305 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:51.861416 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:51.861932 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:51.862722 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:51.863666 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:51.863993 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:51.864116 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:51.866297 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:51.866304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:51.866304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:51.866311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:51.867925 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:51.868299 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:51.868301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:51.868302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:51.868409 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:51.868671 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:51.868672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:51.868672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:51.868866 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:51.868936 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:51.869095 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:11:51.869180 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
I20260619 12:11:51.869401 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.860484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41351 (local address 127.0.0.1:50434)
0619 12:11:51.860648 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.860688 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:51.861132 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:51.861423 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:51.861425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:51.861659 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:51.861809 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:51.861813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:51.862833 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:51.862834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:51.863574 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:51.863576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:51.864089 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:51.864090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:51.864152 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:51.864669 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:51.864677 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:51.866199 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:51.868021 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:51.868027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:51.868029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:51.868208 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:51.868479 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:51.868480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:51.868481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:51.868602 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:51.868928 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:51.868932 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:51.868987 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 12:11:51.869061 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:11:51.869255 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":78,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.61@KRBTEST.COM" created.
I20260619 12:11:51.870214 14165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:51.870544 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.870425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42686)
0619 12:11:51.870476 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.870517 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 12:11:51.870693 14116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
I20260619 12:11:51.871062 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.871002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42698)
0619 12:11:51.871026 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.871048 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 12:11:51.871171 14165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:32805: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab.
Entry for principal kudu/127.0.201.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab.
I20260619 12:11:51.884095 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:32805
--webserver_interface=127.0.201.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab
--principal=kudu/127.0.201.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:41351,127.0.201.61:32805,127.0.201.60:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:51.920418 14165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } attempt: 1
I20260619 12:11:51.920986 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.920814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42706)
0619 12:11:51.920907 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.920953 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 12:11:51.921391 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.921297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42712)
0619 12:11:51.921339 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.921365 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:11:51.921499 14165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:32805: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
W20260619 12:11:51.963490 14172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:51.963677 14172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:51.963698 14172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:51.964736 14172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:51.964769 14172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:51.964783 14172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:51.964797 14172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:51.964807 14172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:51.966183 14172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:41351,127.0.201.61:32805,127.0.201.60:41817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab
--principal=kudu/127.0.201.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.201.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:51.966359 14172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:51.966511 14172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:51.971272 14177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:51.971272 14180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:51.971302 14178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:51.971333 14172 server_base.cc:1034] running on GCE node
I20260619 12:11:51.971719 14172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:51.972039 14172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:51.973249 14172 hybrid_clock.cc:648] HybridClock initialized: now 1781871111973235 us; error 24 us; skew 500 ppm
Jun 19 12:11:51 dist-test-slave-c910 krb5kdc[14084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:51.975740 14172 init.cc:377] Logged in from keytab as kudu/127.0.201.61@KRBTEST.COM (short username kudu)
I20260619 12:11:51.976182 14172 webserver.cc:460] Webserver started at http://127.0.201.61:42347/ using document root <none> and password file <none>
I20260619 12:11:51.976416 14172 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:51.976485 14172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:51.976636 14172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:51.977721 14172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data/instance:
uuid: "76381c0153094ede98c866529fd858fd"
format_stamp: "Formatted at 2026-06-19 12:11:51 on dist-test-slave-c910"
server_key: "55d490892c1ac8c2d3f76273e387067c"
server_key_iv: "868a281a269e0d017f5a54edca228b5b"
server_key_version: "encryptionkey@0"
I20260619 12:11:51.978101 14172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal/instance:
uuid: "76381c0153094ede98c866529fd858fd"
format_stamp: "Formatted at 2026-06-19 12:11:51 on dist-test-slave-c910"
server_key: "55d490892c1ac8c2d3f76273e387067c"
server_key_iv: "868a281a269e0d017f5a54edca228b5b"
server_key_version: "encryptionkey@0"
I20260619 12:11:51.979445 14172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:11:51.980149 14187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:51.980322 14172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:51.980418 14172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal
uuid: "76381c0153094ede98c866529fd858fd"
format_stamp: "Formatted at 2026-06-19 12:11:51 on dist-test-slave-c910"
server_key: "55d490892c1ac8c2d3f76273e387067c"
server_key_iv: "868a281a269e0d017f5a54edca228b5b"
server_key_version: "encryptionkey@0"
I20260619 12:11:51.980497 14172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:51.987839 14172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:51.989639 14165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } attempt: 2
I20260619 12:11:51.990008 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.989891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42728)
0619 12:11:51.989948 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.989983 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:11:51.990176 14172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:51.990319 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.990226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42730)
0619 12:11:51.990269 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.990292 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 12:11:51.990334 14172 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260619 12:11:51.990394 14165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:32805: Network error: Client connection negotiation failed: client connection to 127.0.201.61:32805: connect: Connection refused (error 111)
I20260619 12:11:51.994333 14172 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.61:32805
I20260619 12:11:51.994396 14239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.61:32805 every 8 connection(s)
I20260619 12:11:51.994725 14172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
I20260619 12:11:51.995630 14240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:51.997083 14240 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:51.998399 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14172
I20260619 12:11:51.998503 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-1/wal/instance
I20260619 12:11:51.998723 804 external_mini_cluster.cc:1428] Setting key 7ffebaa30630e2e8f9dd4859c9ad2c56
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:52.006774 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.997738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.0.201.62:41351)
0619 12:11:51.997811 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:51.997814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:51.998041 (+ 227us) server_negotiation.cc:407] Connection header received
0619 12:11:51.998605 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:51.998607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:51.998623 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:51.998658 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:51.999652 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.000135 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.001177 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.001519 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.001591 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.003754 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.003756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.003756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.003765 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.005445 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.005820 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.005821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.005822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.005882 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.006111 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.006112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.006113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.006268 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.006350 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.006582 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.006670 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:11:52.006843 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:51.997685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41351 (local address 127.0.0.1:50440)
0619 12:11:51.997880 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:51.997925 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:51.998472 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:51.998839 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:51.998844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:51.999269 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:51.999550 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:51.999556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.000263 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.000264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.001071 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.001073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.001603 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.001604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.001718 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.002226 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.002233 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.003664 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.005530 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.005535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.005537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.005741 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.005957 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.005957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.005958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.006045 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.006341 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.006345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.006454 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.006545 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.006660 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.0.201.60@KRBTEST.COM; defaulting to no policy
I20260619 12:11:52.007444 14240 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.60@KRBTEST.COM" created.
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:11:52.015020 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.007789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.0.201.61:32805)
0619 12:11:52.007944 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.007948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.007962 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:11:52.007977 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.007979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.007988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.008071 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.008322 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.009069 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.009874 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.010313 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.010390 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.012215 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.012222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.012222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.012228 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.013782 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.014212 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.014214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.014215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.014264 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.014476 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.014477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.014477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.014620 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.014677 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.014858 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.014938 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260619 12:11:52.015159 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.007704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42736)
0619 12:11:52.007751 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.007767 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.007797 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.008083 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.008084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.008207 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.008250 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.008251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.009151 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.009152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.009796 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.009799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.010417 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.010418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.010481 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.010907 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.010911 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.012121 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.013866 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.013867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.013867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.014124 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.014328 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.014329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.014330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.014407 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.014687 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.014688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.014741 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.014838 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.015002 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":53}
I20260619 12:11:52.015867 14240 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:52.016258 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.016116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55362)
0619 12:11:52.016178 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.016230 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 12:11:52.016405 14189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
I20260619 12:11:52.016878 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.016766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55376)
0619 12:11:52.016818 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.016848 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 12:11:52.016995 14240 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41817: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab.
Entry for principal kudu/127.0.201.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab.
I20260619 12:11:52.026458 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:41817
--webserver_interface=127.0.201.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab
--principal=kudu/127.0.201.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:41351,127.0.201.61:32805,127.0.201.60:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:52.066262 14240 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } attempt: 1
I20260619 12:11:52.066859 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.066673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55390)
0619 12:11:52.066752 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.066813 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 12:11:52.067302 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.067177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55398)
0619 12:11:52.067248 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.067279 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260619 12:11:52.067420 14240 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41817: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
I20260619 12:11:52.081542 14165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } attempt: 3
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:11:52.089512 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.081902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42744)
0619 12:11:52.081963 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.081978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.082030 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.082233 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.082234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.082383 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.082450 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.082451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.083610 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.083612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.084281 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.084284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.084817 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.084818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.084879 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.085356 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.085361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.086860 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.088602 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.088603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.088603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.088760 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.088969 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.088970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.088970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.089018 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.089275 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.089276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.089313 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.089362 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.089438 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:11:52.089571 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.081997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.0.201.61:32805)
0619 12:11:52.082079 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.082082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.082093 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:11:52.082123 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.082124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.082136 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.082230 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.082586 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.083494 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.084372 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.084741 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.084822 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.086970 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.086972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.086972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.086981 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.088522 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.088833 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.088834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.088834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.088897 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.089077 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.089078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.089078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.089214 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.089284 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.089388 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.089469 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:11:52.089985 14165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:52.090365 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.090265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55404)
0619 12:11:52.090312 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.090341 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:11:52.090662 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.090596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55418)
0619 12:11:52.090627 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.090648 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 12:11:52.090761 14165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41817: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
W20260619 12:11:52.114109 14247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:52.114300 14247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:52.114322 14247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:52.115540 14247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:52.115581 14247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:52.115597 14247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:52.115639 14247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:52.115657 14247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:52.117113 14247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:41351,127.0.201.61:32805,127.0.201.60:41817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab
--principal=kudu/127.0.201.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:41817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.201.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:52.117353 14247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:52.117575 14247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:11:52.121929 14165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } attempt: 1
I20260619 12:11:52.122390 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.122268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55434)
0619 12:11:52.122326 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.122353 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:11:52.122426 14255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:52.122434 14253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:52.122534 14247 server_base.cc:1034] running on GCE node
W20260619 12:11:52.122596 14252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:52.122870 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.122748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55446)
0619 12:11:52.122810 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.122843 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 12:11:52.122970 14247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260619 12:11:52.123087 14165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41817: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
I20260619 12:11:52.123311 14247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:52.124524 14247 hybrid_clock.cc:648] HybridClock initialized: now 1781871112124516 us; error 37 us; skew 500 ppm
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:52.127161 14247 init.cc:377] Logged in from keytab as kudu/127.0.201.60@KRBTEST.COM (short username kudu)
I20260619 12:11:52.127707 14247 webserver.cc:460] Webserver started at http://127.0.201.60:44277/ using document root <none> and password file <none>
I20260619 12:11:52.127899 14247 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:52.127945 14247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:52.128033 14247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:52.129078 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data/instance:
uuid: "33db9377b5a14987b5345b2bbd9b5fb4"
format_stamp: "Formatted at 2026-06-19 12:11:52 on dist-test-slave-c910"
server_key: "286a37051db923a4a9696e543b6372b5"
server_key_iv: "da9751a1928f8177a87ffba4be47ea17"
server_key_version: "encryptionkey@0"
I20260619 12:11:52.129423 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal/instance:
uuid: "33db9377b5a14987b5345b2bbd9b5fb4"
format_stamp: "Formatted at 2026-06-19 12:11:52 on dist-test-slave-c910"
server_key: "286a37051db923a4a9696e543b6372b5"
server_key_iv: "da9751a1928f8177a87ffba4be47ea17"
server_key_version: "encryptionkey@0"
I20260619 12:11:52.130759 14247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:11:52.131592 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:52.131803 14247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:52.131876 14247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal
uuid: "33db9377b5a14987b5345b2bbd9b5fb4"
format_stamp: "Formatted at 2026-06-19 12:11:52 on dist-test-slave-c910"
server_key: "286a37051db923a4a9696e543b6372b5"
server_key_iv: "da9751a1928f8177a87ffba4be47ea17"
server_key_version: "encryptionkey@0"
I20260619 12:11:52.132006 14247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:52.135591 14240 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } attempt: 2
I20260619 12:11:52.135962 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.135834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55458)
0619 12:11:52.135888 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.135930 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 12:11:52.136304 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.136222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55472)
0619 12:11:52.136264 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.136288 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:11:52.136420 14240 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41817: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41817: connect: Connection refused (error 111)
I20260619 12:11:52.160187 14247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:52.162118 14247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:52.162290 14247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:52.166386 14247 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.60:41817
I20260619 12:11:52.166420 14314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.60:41817 every 8 connection(s)
I20260619 12:11:52.166776 14247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
I20260619 12:11:52.167688 14315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:52.169137 14315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:52.171492 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14247
I20260619 12:11:52.171612 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781870963166527-804-0/minicluster-data/master-2/wal/instance
I20260619 12:11:52.171814 804 external_mini_cluster.cc:1428] Setting key 02401d2f3793098e8343447e1149589f
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:11:52.179265 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.169777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.0.201.62:41351)
0619 12:11:52.169835 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.169836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.170027 (+ 191us) server_negotiation.cc:407] Connection header received
0619 12:11:52.170471 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.170472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.170481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.170542 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.171119 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.171690 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.172973 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.173280 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.173332 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.175556 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.175557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.175557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.175564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.177636 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.178110 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.178111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.178111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.178168 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.178468 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.178469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.178469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.178633 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.178705 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.179056 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.179142 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:11:52.179417 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.169744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:41351 (local address 127.0.0.1:50450)
0619 12:11:52.169892 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.169932 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.170357 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.170598 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.170601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.170851 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.171047 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.171051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.171784 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.171786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.172887 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.172888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.173355 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.173356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.173552 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.174023 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.174032 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.175473 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.177715 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.177722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.177724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.178039 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.178243 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.178245 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.178245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.178383 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.178713 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.178718 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.178874 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.179022 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.179148 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260619 12:11:52.179986 14315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:52.180962 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.172200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.0.201.60:41817)
0619 12:11:52.172348 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.172352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.172412 (+ 60us) server_negotiation.cc:407] Connection header received
0619 12:11:52.172471 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.172473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.172494 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.172567 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.173177 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.174195 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.174920 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.175319 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.175409 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.177909 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.177915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.177916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.177921 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.179825 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.180177 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.180179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.180180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.180226 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.180451 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.180452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.180452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.180621 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.180774 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.180843 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.180900 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:52.187287 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.180275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:32805 (local address 127.0.0.1:42754)
0619 12:11:52.180346 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.180355 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.180401 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.180539 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.180539 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.180683 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.180712 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.180714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.181552 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.181554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.182346 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.182348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.182775 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.182776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.182816 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.183218 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.183223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.184504 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.186022 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.186023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.186024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.186170 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.186377 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.186378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.186378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.186453 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.186912 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.186912 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.186945 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.187086 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.187174 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 12:11:52.187311 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.180309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.0.201.61:32805)
0619 12:11:52.180360 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.180362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.180388 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:11:52.180468 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.180469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.180480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.180528 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.180783 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.181465 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.182419 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.182701 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.182773 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.184594 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.184595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.184595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.184602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.185938 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.186239 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.186241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.186241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.186299 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.186537 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.186538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.186539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.186639 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.186716 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.187116 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.187187 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:11:52.187687 14315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:11:52.189670 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.181717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.0.201.62:41351)
0619 12:11:52.181750 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.181751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.181759 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:11:52.181780 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.181781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.181791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.181837 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.182356 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.183069 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.183987 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.184287 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.184360 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.186725 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.186726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.186727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.186733 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.188486 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.188871 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.188872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.188873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.188934 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.189129 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.189130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.189131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.189234 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.189466 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.189509 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.189586 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:11:52.194912 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.187895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55478)
0619 12:11:52.188566 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.188575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.188606 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.188726 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.188726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.188822 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.188848 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.188849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.189436 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.189437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.190171 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.190174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.190631 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.190632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.190673 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.191075 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.191080 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.192213 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.193895 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.193896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.193896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.194145 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.194314 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.194315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.194315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.194393 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.194616 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.194617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.194671 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.194754 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.194852 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658}
I20260619 12:11:52.195020 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.187957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.0.201.60:41817)
0619 12:11:52.187990 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.187992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.188623 (+ 631us) server_negotiation.cc:407] Connection header received
0619 12:11:52.188646 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.188647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.188657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.188726 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.188920 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.189362 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.190253 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.190557 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.190628 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.192295 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.192296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.192297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.192304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.193811 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.194226 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.194227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.194227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.194270 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.194461 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.194462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.194462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.194548 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.194591 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.194788 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.194907 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":16}
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:11:52.197068 14315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: Bootstrap starting.
I20260619 12:11:52.198113 14315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:52.198216 14165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } attempt: 2
I20260619 12:11:52.198489 14315 log.cc:824] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:52.198521 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.190237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.0.201.61:32805)
0619 12:11:52.190268 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.190269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.190278 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:11:52.190310 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.190310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.190320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.190361 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.190889 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.191600 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.192858 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.193042 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.193106 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.195815 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.195816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.195817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.195823 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.197270 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.197569 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.197570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.197571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.197643 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.197815 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.197816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.197816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.197935 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.198104 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.198145 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.198430 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 12:11:52.199555 14315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: No bootstrap required, opened a new log
I20260619 12:11:52.199918 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:11:52.201635 14315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.201886 14315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:52.201988 14315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33db9377b5a14987b5345b2bbd9b5fb4, State: Initialized, Role: FOLLOWER
I20260619 12:11:52.202190 14315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:11:52.202637 14315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 12:11:52.204495 14340 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:11:52.205060 14340 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:11:52.204692 14328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.205240 14328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:11:52.205885 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.198456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55480)
0619 12:11:52.198502 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.198512 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.198548 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.198699 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.198699 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.198850 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.198890 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.198892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.199703 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.199704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.200429 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.200432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.200929 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.200930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.200986 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.201401 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.201406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.202769 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.204633 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.204634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.204635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.204834 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.205234 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.205234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.205235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.205293 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.205596 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.205597 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.205637 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.205700 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.205787 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 12:11:52.206146 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.198492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.0.201.60:41817)
0619 12:11:52.198543 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.198544 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.198552 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:11:52.198606 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.198607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.198616 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.198694 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.198961 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.199638 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.200525 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.200818 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.200925 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.202871 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.202872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.202872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.202878 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.204540 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.205109 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.205110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.205110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.205165 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.205356 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.205357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.205357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.205496 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.205569 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.205998 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.206070 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:11:52.205492 14247 master_runner.cc:186] Error getting master registration for 127.0.201.62:41351: OK, instance_id { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" instance_seqno: 1781871111855686 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:11:52.207845 14247 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:11:52.209523 14165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3: Bootstrap starting.
I20260619 12:11:52.210258 14165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:52.210552 14165 log.cc:824] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:52.211606 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.201243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.0.201.62:41351)
0619 12:11:52.201278 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.201280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.201492 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:11:52.201603 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.201605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.201615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.201645 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.202399 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.203709 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.207030 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.207333 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.207373 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.208856 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.208858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.208858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.208865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.210403 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.210775 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.210775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.210776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.210830 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.211014 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.211015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.211015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.211126 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.211219 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.211432 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.211522 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:11:52.212987 14165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3: No bootstrap required, opened a new log
I20260619 12:11:52.213218 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.201535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.0.201.60:41817)
0619 12:11:52.201765 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.201767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.201920 (+ 153us) server_negotiation.cc:407] Connection header received
0619 12:11:52.201939 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.201940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.201948 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.202007 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.202671 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.203418 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.204536 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.204842 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.204907 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.207122 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.207123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.207123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.207129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.211590 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.212239 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.212240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.212241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.212329 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.212570 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.212571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.212571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.212698 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.212778 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.212986 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.213075 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":70,"threads_started":1}
I20260619 12:11:52.215024 14165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.215152 14165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:52.215188 14165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db40b5b15c0431487330af8fd178ce3, State: Initialized, Role: FOLLOWER
I20260619 12:11:52.215324 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.201326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.0.201.61:32805)
0619 12:11:52.201363 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.201364 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.205081 (+ 3717us) server_negotiation.cc:407] Connection header received
0619 12:11:52.208149 (+ 3068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.208151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.208162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.208193 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.208999 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.209750 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.210853 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.211154 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.211232 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.212740 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.212741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.212741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.212747 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.214185 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.214529 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.214541 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.214542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.214600 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.214795 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.214796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.214797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.214903 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.214995 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.215136 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.215228 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:11:52.215417 14165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.215749 14165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 12:11:52.217653 14354 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:11:52.217782 14354 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:11:52.218021 14343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.218104 14343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:11:52.218597 14097 master_runner.cc:186] Error getting master registration for 127.0.201.61:32805: OK, instance_id { permanent_uuid: "76381c0153094ede98c866529fd858fd" instance_seqno: 1781871111993201 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:11:52.220223 14097 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:11:52.227560 14240 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } attempt: 3
Jun 19 12:11:52 dist-test-slave-c910 krb5kdc[14084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:11:52.232451 14343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:11:52.232545 14343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.232779 14343 leader_election.cc:288] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76381c0153094ede98c866529fd858fd (127.0.201.61:32805), 33db9377b5a14987b5345b2bbd9b5fb4 (127.0.201.60:41817)
I20260619 12:11:52.233093 14289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db40b5b15c0431487330af8fd178ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" is_pre_election: true
I20260619 12:11:52.233232 14289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2db40b5b15c0431487330af8fd178ce3 in term 0.
I20260619 12:11:52.233464 14114 leader_election.cc:302] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db40b5b15c0431487330af8fd178ce3, 33db9377b5a14987b5345b2bbd9b5fb4; no voters:
I20260619 12:11:52.233604 14343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:11:52.233683 14343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:11:52.233713 14343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:52.234576 14343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.234683 14343 leader_election.cc:288] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [CANDIDATE]: Term 1 election: Requested vote from peers 76381c0153094ede98c866529fd858fd (127.0.201.61:32805), 33db9377b5a14987b5345b2bbd9b5fb4 (127.0.201.60:41817)
I20260619 12:11:52.234848 14242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.227803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41817 (local address 127.0.0.1:55496)
0619 12:11:52.227850 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:52.227861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:52.227899 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:52.228029 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:52.228030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:52.228179 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:52.228213 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.228215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.228941 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.228942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.229532 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:52.229533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.229993 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.229994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:52.230057 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.230518 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:52.230523 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:52.231778 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:52.233785 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.233786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.233786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.234001 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.234192 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:52.234194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:52.234194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:52.234250 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:52.234528 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:52.234529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:52.234571 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 12:11:52.234657 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:11:52.234751 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 12:11:52.234987 14289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db40b5b15c0431487330af8fd178ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33db9377b5a14987b5345b2bbd9b5fb4"
I20260619 12:11:52.235419 14289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:52.236264 14289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2db40b5b15c0431487330af8fd178ce3 in term 1.
I20260619 12:11:52.236359 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:52.227864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55496 (local address 127.0.201.60:41817)
0619 12:11:52.227901 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:52.227903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:52.227911 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:11:52.227957 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:52.227958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:52.227968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:52.228034 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:52.228293 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.228863 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.229654 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:52.229923 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:52.229996 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:52.231870 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:52.231872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:52.231872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:52.231879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:52.233711 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.234066 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.234067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.234068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.234118 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:52.234321 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:52.234322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:52.234322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:52.234468 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:52.234529 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:52.234692 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 12:11:52.234762 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:11:52.236428 14114 leader_election.cc:302] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db40b5b15c0431487330af8fd178ce3, 33db9377b5a14987b5345b2bbd9b5fb4; no voters:
I20260619 12:11:52.236500 14343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:52.236630 14343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [term 1 LEADER]: Becoming Leader. State: Replica: 2db40b5b15c0431487330af8fd178ce3, State: Running, Role: LEADER
I20260619 12:11:52.236706 14343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.237058 14343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db40b5b15c0431487330af8fd178ce3. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.237118 14343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:52.237347 14358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:52.237684 14240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: Bootstrap starting.
I20260619 12:11:52.237793 14358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:52.238392 14240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:52.238692 14240 log.cc:824] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:52.239576 14240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: No bootstrap required, opened a new log
I20260619 12:11:52.240505 14289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [term 1 FOLLOWER]: Refusing update from remote peer 2db40b5b15c0431487330af8fd178ce3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:52.240840 14356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:52.241632 14240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.241760 14240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:52.241791 14240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76381c0153094ede98c866529fd858fd, State: Initialized, Role: FOLLOWER
I20260619 12:11:52.241905 14240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } }
I20260619 12:11:52.242174 14240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:52.242183 14328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db40b5b15c0431487330af8fd178ce3. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.242249 14328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:11:52.242244 14212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:52.243067 14212 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [term 1 FOLLOWER]: Refusing update from remote peer 2db40b5b15c0431487330af8fd178ce3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:52.243431 14343 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:52.243748 14328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.243819 14328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:11:52.243862 14213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db40b5b15c0431487330af8fd178ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76381c0153094ede98c866529fd858fd"
I20260619 12:11:52.243875 14356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.243940 14356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:52.244179 14360 mvcc.cc:205] Tried to move back new op lower bound from 7298544075734786048 to 7298544075722039296. Current Snapshot: MvccSnapshot[applied={T|T < 7298544075734786048}]
I20260619 12:11:52.244628 14213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2db40b5b15c0431487330af8fd178ce3 in term 1.
I20260619 12:11:52.244937 14214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db40b5b15c0431487330af8fd178ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76381c0153094ede98c866529fd858fd" is_pre_election: true
I20260619 12:11:52.245404 14361 mvcc.cc:205] Tried to move back new op lower bound from 7298544075734786048 to 7298544075722039296. Current Snapshot: MvccSnapshot[applied={T|T < 7298544075734786048}]
I20260619 12:11:52.245573 14358 catalog_manager.cc:1347] Generated new cluster ID: ab6b8dbf82fe4bdb8a39c9e8495524e8
I20260619 12:11:52.245647 14358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:11:52.246070 14362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.246217 14362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:11:52.246301 14362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.246389 14362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:11:52.245668 14380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db40b5b15c0431487330af8fd178ce3. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.247342 14380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:11:52.251827 14375 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:11:52.251924 14375 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:11:52.274812 14356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2db40b5b15c0431487330af8fd178ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db40b5b15c0431487330af8fd178ce3" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 41351 } } peers { permanent_uuid: "76381c0153094ede98c866529fd858fd" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 32805 } } peers { permanent_uuid: "33db9377b5a14987b5345b2bbd9b5fb4" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41817 } } }
I20260619 12:11:52.274940 14356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:52.276661 14358 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:11:52.277441 14358 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:52.287011 14358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2db40b5b15c0431487330af8fd178ce3: Generated new TSK 0
I20260619 12:11:52.287256 14358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:11:53.205565 14340 catalog_manager.cc:1259] Loaded cluster ID: ab6b8dbf82fe4bdb8a39c9e8495524e8
I20260619 12:11:53.205647 14340 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: loading cluster ID for follower catalog manager: success
I20260619 12:11:53.206545 14340 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: acquiring CA information for follower catalog manager: success
I20260619 12:11:53.206828 14340 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33db9377b5a14987b5345b2bbd9b5fb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:11:53.252367 14375 catalog_manager.cc:1259] Loaded cluster ID: ab6b8dbf82fe4bdb8a39c9e8495524e8
I20260619 12:11:53.252444 14375 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: loading cluster ID for follower catalog manager: success
I20260619 12:11:53.253281 14375 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: acquiring CA information for follower catalog manager: success
I20260619 12:11:53.253603 14375 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 76381c0153094ede98c866529fd858fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:11:53.343003 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14097
I20260619 12:11:53.348551 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14172
I20260619 12:11:53.353708 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14247
2026-06-19T12:11:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2780 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:11:53 dist-test-slave-c910 krb5kdc[14391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:11:53 dist-test-slave-c910 krb5kdc[14391](info): set up 2 sockets
Jun 19 12:11:53 dist-test-slave-c910 krb5kdc[14391](info): commencing operation
krb5kdc: starting...
W20260619 12:11:54.475391 804 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:11:54 dist-test-slave-c910 krb5kdc[14391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:11:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:11:54.591279 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38805
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:38805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:54.669400 14404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:54.669646 14404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:54.669682 14404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:54.670796 14404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:54.670863 14404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:54.670887 14404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:54.670910 14404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:54.670938 14404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:54.672415 14404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:38805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:54.672677 14404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:54.672888 14404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:54.677795 14410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:54.677803 14412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:54.677796 14409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:54.677819 14404 server_base.cc:1034] running on GCE node
I20260619 12:11:54.678349 14404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:54.678632 14404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:54.679769 14404 hybrid_clock.cc:648] HybridClock initialized: now 1781871114679761 us; error 35 us; skew 500 ppm
Jun 19 12:11:54 dist-test-slave-c910 krb5kdc[14391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:54.682039 14404 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:54.682498 14404 webserver.cc:460] Webserver started at http://127.0.201.62:44587/ using document root <none> and password file <none>
I20260619 12:11:54.682729 14404 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:54.682794 14404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:54.682940 14404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:54.684015 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "7cb4322d06a94b52809d32da5548c9e8"
format_stamp: "Formatted at 2026-06-19 12:11:54 on dist-test-slave-c910"
server_key: "70c8f77fdfebcb37682f10fabd3070fd"
server_key_iv: "b74000d97dd9a65a12e21db9d202f5b1"
server_key_version: "encryptionkey@0"
I20260619 12:11:54.684360 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "7cb4322d06a94b52809d32da5548c9e8"
format_stamp: "Formatted at 2026-06-19 12:11:54 on dist-test-slave-c910"
server_key: "70c8f77fdfebcb37682f10fabd3070fd"
server_key_iv: "b74000d97dd9a65a12e21db9d202f5b1"
server_key_version: "encryptionkey@0"
I20260619 12:11:54.685685 14404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:11:54.686334 14419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:54.686534 14404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:11:54.686622 14404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "7cb4322d06a94b52809d32da5548c9e8"
format_stamp: "Formatted at 2026-06-19 12:11:54 on dist-test-slave-c910"
server_key: "70c8f77fdfebcb37682f10fabd3070fd"
server_key_iv: "b74000d97dd9a65a12e21db9d202f5b1"
server_key_version: "encryptionkey@0"
I20260619 12:11:54.686674 14404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:54.697275 14404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:54.699087 14404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:54.699235 14404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:54.702759 14404 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:38805
I20260619 12:11:54.702804 14471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:38805 every 8 connection(s)
I20260619 12:11:54.703127 14404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:54.703953 14472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:54.705406 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14404
I20260619 12:11:54.705506 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:11:54.705696 804 external_mini_cluster.cc:1428] Setting key 5ae2dd55f5c1e11d42053ad0971a5ad7
I20260619 12:11:54.706547 14472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8: Bootstrap starting.
I20260619 12:11:54.707250 14472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:54.707548 14472 log.cc:824] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:54.708223 14472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8: No bootstrap required, opened a new log
I20260619 12:11:54.710340 14472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb4322d06a94b52809d32da5548c9e8" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38805 } }
I20260619 12:11:54.710503 14472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:54.710541 14472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb4322d06a94b52809d32da5548c9e8, State: Initialized, Role: FOLLOWER
I20260619 12:11:54.710675 14472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb4322d06a94b52809d32da5548c9e8" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38805 } }
I20260619 12:11:54.710765 14472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:54.710790 14472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:54.710866 14472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:54.711647 14472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb4322d06a94b52809d32da5548c9e8" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38805 } }
I20260619 12:11:54.711768 14472 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb4322d06a94b52809d32da5548c9e8; no voters:
I20260619 12:11:54.711943 14472 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:11:54.711997 14477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:54.712169 14477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [term 1 LEADER]: Becoming Leader. State: Replica: 7cb4322d06a94b52809d32da5548c9e8, State: Running, Role: LEADER
I20260619 12:11:54.712203 14472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:54.712347 14477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb4322d06a94b52809d32da5548c9e8" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38805 } }
I20260619 12:11:54.712740 14478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb4322d06a94b52809d32da5548c9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb4322d06a94b52809d32da5548c9e8" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38805 } } }
Jun 19 12:11:54 dist-test-slave-c910 krb5kdc[14391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:54.712817 14478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:54.713019 14479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb4322d06a94b52809d32da5548c9e8. Latest consensus state: current_term: 1 leader_uuid: "7cb4322d06a94b52809d32da5548c9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb4322d06a94b52809d32da5548c9e8" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38805 } } }
I20260619 12:11:54.713155 14479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:54.713398 14485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:54.713872 14485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:54.716245 14485 catalog_manager.cc:1347] Generated new cluster ID: 9aefa977446a49449ac5b7779a68852a
I20260619 12:11:54.716302 14485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:11:54.717180 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:54.706181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.0.201.62:38805)
0619 12:11:54.706410 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:54.706418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:54.706447 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:11:54.706989 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:54.707006 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:54.707063 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:54.707229 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:54.707977 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:54.708914 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:54.710026 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:54.710397 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:54.710471 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:54.713435 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:54.713439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:54.713439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:54.713452 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:54.715256 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:54.715615 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:54.715618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:54.715619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:54.715725 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:54.715949 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:54.715950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:54.715950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:54.716244 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:54.716380 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:54.716840 (+ 460us) server_negotiation.cc:299] Negotiation successful
0619 12:11:54.716973 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":63,"threads_started":1}
I20260619 12:11:54.723955 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:11:54.731076 14485 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:11:54.731806 14485 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:54.734450 14485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cb4322d06a94b52809d32da5548c9e8: Generated new TSK 0
I20260619 12:11:54.734633 14485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:11:54.777704 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14404
2026-06-19T12:11:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1422 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:11:54 dist-test-slave-c910 krb5kdc[14499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:11:54 dist-test-slave-c910 krb5kdc[14499](info): set up 2 sockets
Jun 19 12:11:54 dist-test-slave-c910 krb5kdc[14499](info): commencing operation
krb5kdc: starting...
W20260619 12:11:55.815356 804 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:11:55 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:11:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:11:55.942078 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46683
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:56.021714 14512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:56.021904 14512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:56.021926 14512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:56.022979 14512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:56.023017 14512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:56.023031 14512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:56.023043 14512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:56.023056 14512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:56.024452 14512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:56.024641 14512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:56.024796 14512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:56.029563 14520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.029569 14517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.029569 14518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.029609 14512 server_base.cc:1034] running on GCE node
I20260619 12:11:56.029971 14512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:56.030208 14512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:56.031389 14512 hybrid_clock.cc:648] HybridClock initialized: now 1781871116031371 us; error 30 us; skew 500 ppm
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:56.033756 14512 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:56.034202 14512 webserver.cc:460] Webserver started at http://127.0.201.62:46689/ using document root <none> and password file <none>
I20260619 12:11:56.034428 14512 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:56.034508 14512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:56.034657 14512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:56.035740 14512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "b15f8925ffcc4226aa7b9adcf2f5803b"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "1724013940101a913fe94685e1822d1d"
server_key_iv: "7513b932d9bc24f4d247df2da4ff0e5c"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.036082 14512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "b15f8925ffcc4226aa7b9adcf2f5803b"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "1724013940101a913fe94685e1822d1d"
server_key_iv: "7513b932d9bc24f4d247df2da4ff0e5c"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.037338 14512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:11:56.038038 14527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.038204 14512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:56.038300 14512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "b15f8925ffcc4226aa7b9adcf2f5803b"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "1724013940101a913fe94685e1822d1d"
server_key_iv: "7513b932d9bc24f4d247df2da4ff0e5c"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.038378 14512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:56.050377 14512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:56.052565 14512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:56.052719 14512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:56.056703 14512 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:46683
I20260619 12:11:56.056733 14579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:46683 every 8 connection(s)
I20260619 12:11:56.057062 14512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:56.057952 14580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:56.060187 14580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap starting.
I20260619 12:11:56.060750 14580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:56.061038 14580 log.cc:824] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:56.061676 14580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: No bootstrap required, opened a new log
I20260619 12:11:56.063171 14580 raft_consensus.cc:348] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.063318 14580 raft_consensus.cc:374] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:56.063398 14580 raft_consensus.cc:729] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15f8925ffcc4226aa7b9adcf2f5803b, State: Initialized, Role: FOLLOWER
I20260619 12:11:56.063575 14580 consensus_queue.cc:260] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.063634 14580 raft_consensus.cc:388] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:56.063686 14580 raft_consensus.cc:482] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:56.063771 14580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:56.064560 14580 raft_consensus.cc:504] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.064670 14580 leader_election.cc:302] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b15f8925ffcc4226aa7b9adcf2f5803b; no voters:
I20260619 12:11:56.064814 14580 leader_election.cc:288] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:11:56.064839 14583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:56.064998 14583 raft_consensus.cc:686] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 LEADER]: Becoming Leader. State: Replica: b15f8925ffcc4226aa7b9adcf2f5803b, State: Running, Role: LEADER
I20260619 12:11:56.065075 14580 sys_catalog.cc:564] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:56.065127 14583 consensus_queue.cc:237] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.065506 14584 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } } }
I20260619 12:11:56.065603 14584 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:56.065508 14585 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b15f8925ffcc4226aa7b9adcf2f5803b. Latest consensus state: current_term: 1 leader_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } } }
I20260619 12:11:56.065852 14592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:56.065866 14585 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:56.066344 14592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:56.066423 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14512
I20260619 12:11:56.066540 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:11:56.066751 804 external_mini_cluster.cc:1428] Setting key 3d0e2b136a3a30bb15c36cafcba80737
I20260619 12:11:56.067924 14592 catalog_manager.cc:1347] Generated new cluster ID: e8bed7320c7246d398aeb96abe673b1f
I20260619 12:11:56.067973 14592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:56.077080 14592 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:11:56.077648 14592 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:56.078323 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.067170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.0.201.62:46683)
0619 12:11:56.067661 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.067671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.067700 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:11:56.067811 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.067818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.067882 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.068011 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:56.068746 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.070157 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.071164 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.071842 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.071963 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.074663 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:56.074667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:56.074667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:56.074678 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:56.076598 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.077138 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.077141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.077141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.077246 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.077466 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.077467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.077467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.077738 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:56.077894 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.077962 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.078103 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":87,"threads_started":1}
I20260619 12:11:56.079873 14592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Generated new TSK 0
I20260619 12:11:56.080034 14592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:11:56.109951 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46683
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:56.202631 14606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:56.202850 14606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:56.202888 14606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:56.204102 14606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:56.204161 14606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:56.204205 14606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:11:56.205641 14606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:56.205886 14606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:56.206106 14606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:56.211906 14611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.211916 14612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.211917 14614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.212076 14606 server_base.cc:1034] running on GCE node
I20260619 12:11:56.212339 14606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:56.212589 14606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:56.213740 14606 hybrid_clock.cc:648] HybridClock initialized: now 1781871116213723 us; error 38 us; skew 500 ppm
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:56.216454 14606 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:11:56.217010 14606 webserver.cc:460] Webserver started at http://127.0.201.1:43265/ using document root <none> and password file <none>
I20260619 12:11:56.217252 14606 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:56.217311 14606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:56.217442 14606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:56.218660 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "39b31ed224e74f2fac1396f7665b5b8f"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "555007d0bf3cabce02fdd45f5b91940d"
server_key_iv: "c911a07d85c92125ebb61e9c644495bf"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.219038 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "39b31ed224e74f2fac1396f7665b5b8f"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "555007d0bf3cabce02fdd45f5b91940d"
server_key_iv: "c911a07d85c92125ebb61e9c644495bf"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.220608 14606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:56.221339 14621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.221517 14606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:56.221612 14606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "39b31ed224e74f2fac1396f7665b5b8f"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "555007d0bf3cabce02fdd45f5b91940d"
server_key_iv: "c911a07d85c92125ebb61e9c644495bf"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.221700 14606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:56.254081 14606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:56.255946 14606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:56.256114 14606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:56.256350 14606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:56.256649 14606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:56.256702 14606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.256752 14606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:56.256779 14606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.262513 14606 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:36003
I20260619 12:11:56.262563 14734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:36003 every 8 connection(s)
I20260619 12:11:56.262873 14606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:11:56.264755 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14606
I20260619 12:11:56.264880 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:11:56.265146 804 external_mini_cluster.cc:1428] Setting key 7f7a2dfa951681e428d7fe7571bbbe27
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:56.273182 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.263593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:60817 (local address 127.0.201.62:46683)
0619 12:11:56.263660 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.263662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.263838 (+ 176us) server_negotiation.cc:407] Connection header received
0619 12:11:56.264289 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.264290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.264299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.264353 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:56.265453 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.266105 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.267471 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.267671 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.267728 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.270122 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:56.270123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:56.270124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:56.270131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:56.271731 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.272109 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.272111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.272111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.272189 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.272493 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.272494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.272494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.272632 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:56.272703 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.272959 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.273057 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:11:56.273233 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.263567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:60817)
0619 12:11:56.263708 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.263747 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.264206 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.264405 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.264409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.265103 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:56.265357 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.265362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.266212 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.266214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.267383 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.267387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.267755 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.267757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.267926 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.268437 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:56.268445 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:56.270026 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:56.271804 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:56.271810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:56.271812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:56.272036 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:56.272289 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:56.272290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:56.272290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:56.272407 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:56.272702 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:56.272707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:56.272826 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.272912 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.273055 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
I20260619 12:11:56.273877 14735 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46683
I20260619 12:11:56.273976 14735 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:56.274160 14735 heartbeater.cc:511] Master 127.0.201.62:46683 requested a full tablet report, sending...
I20260619 12:11:56.274688 14544 ts_manager.cc:194] Registered new tserver with Master: 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003)
I20260619 12:11:56.275530 14544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:60817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:11:56.290099 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46683
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:56.372812 14740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:56.373009 14740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:56.373045 14740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:56.374506 14740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:56.374549 14740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:56.374575 14740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:11:56.376260 14740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:56.376500 14740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:56.376726 14740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:56.381541 14746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.381786 14740 server_base.cc:1034] running on GCE node
W20260619 12:11:56.381558 14745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.381541 14748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.382117 14740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:56.382341 14740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:56.383505 14740 hybrid_clock.cc:648] HybridClock initialized: now 1781871116383499 us; error 33 us; skew 500 ppm
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:56.385890 14740 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:11:56.386323 14740 webserver.cc:460] Webserver started at http://127.0.201.2:33637/ using document root <none> and password file <none>
I20260619 12:11:56.386569 14740 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:56.386615 14740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:56.386750 14740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:56.387806 14740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "8f90cda8f49a472388a42c2e9a36cced"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "98167e40c8cb7ff0462f29103c5d2a0e"
server_key_iv: "6d1380b69b314b63968ca0980de6cad0"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.388134 14740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "8f90cda8f49a472388a42c2e9a36cced"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "98167e40c8cb7ff0462f29103c5d2a0e"
server_key_iv: "6d1380b69b314b63968ca0980de6cad0"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.389361 14740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:11:56.390089 14755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.390262 14740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:56.390357 14740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "8f90cda8f49a472388a42c2e9a36cced"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "98167e40c8cb7ff0462f29103c5d2a0e"
server_key_iv: "6d1380b69b314b63968ca0980de6cad0"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.390431 14740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:56.398548 14740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:56.400318 14740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:56.400465 14740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:56.400666 14740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:56.400955 14740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:56.401013 14740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.401050 14740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:56.401078 14740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.406832 14740 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44593
I20260619 12:11:56.406896 14868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44593 every 8 connection(s)
I20260619 12:11:56.407274 14740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:56.414511 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14740
I20260619 12:11:56.414659 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:11:56.414870 804 external_mini_cluster.cc:1428] Setting key b23c546ae2e155da6c05033a16770024
I20260619 12:11:56.416544 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.407977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:33543 (local address 127.0.201.62:46683)
0619 12:11:56.408059 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.408062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.408232 (+ 170us) server_negotiation.cc:407] Connection header received
0619 12:11:56.408647 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.408648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.408658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.408717 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:56.409533 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.410073 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.411026 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.411295 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.411390 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.413355 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:56.413356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:56.413356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:56.413364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:56.415075 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.415576 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.415577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.415578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.415641 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.415947 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.415948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.415948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.416053 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:56.416104 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.416421 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.416482 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:11:56.416853 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.407945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:33543)
0619 12:11:56.408113 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.408145 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.408565 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.408765 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.408768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.409285 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:56.409453 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.409457 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.410157 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.410157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.410919 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.410921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.411380 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.411381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.411482 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.411940 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:56.411948 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:56.413261 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:56.415181 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:56.415188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:56.415190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:56.415504 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:56.415714 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:56.415715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:56.415715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:56.415860 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:56.416118 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:56.416123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:56.416290 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.416404 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.416574 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260619 12:11:56.417371 14869 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46683
I20260619 12:11:56.417479 14869 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:56.417665 14869 heartbeater.cc:511] Master 127.0.201.62:46683 requested a full tablet report, sending...
I20260619 12:11:56.417999 14544 ts_manager.cc:194] Registered new tserver with Master: 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
I20260619 12:11:56.418381 14544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:33543
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:11:56.438515 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46683
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:56.517032 14874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:56.517239 14874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:56.517278 14874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:56.518397 14874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:56.518460 14874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:56.518503 14874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:11:56.519958 14874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:56.520197 14874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:56.520421 14874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:56.525239 14879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.525249 14882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.525313 14874 server_base.cc:1034] running on GCE node
W20260619 12:11:56.525224 14880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.525627 14874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:56.525807 14874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:56.526939 14874 hybrid_clock.cc:648] HybridClock initialized: now 1781871116526927 us; error 28 us; skew 500 ppm
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:56.529443 14874 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:11:56.529793 14874 webserver.cc:460] Webserver started at http://127.0.201.3:38519/ using document root <none> and password file <none>
I20260619 12:11:56.529971 14874 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:56.530022 14874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:56.530103 14874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:11:56.531028 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "de9ddd93604345dba2de5001ed497449"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "86267b2c3bb489646ca0b7285c017367"
server_key_iv: "3b74a92ca976bfd9fa6622b3939da621"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.531384 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "de9ddd93604345dba2de5001ed497449"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "86267b2c3bb489646ca0b7285c017367"
server_key_iv: "3b74a92ca976bfd9fa6622b3939da621"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.532593 14874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:56.533295 14889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.533493 14874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:56.533587 14874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "de9ddd93604345dba2de5001ed497449"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "86267b2c3bb489646ca0b7285c017367"
server_key_iv: "3b74a92ca976bfd9fa6622b3939da621"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.533638 14874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:56.556685 14874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:56.558442 14874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:56.558552 14874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:56.558732 14874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:56.558998 14874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:56.559028 14874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.559051 14874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:56.559062 14874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.564594 14874 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:44481
I20260619 12:11:56.564699 15002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:44481 every 8 connection(s)
I20260619 12:11:56.565012 14874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:56.572909 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 14874
I20260619 12:11:56.573057 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:11:56.573236 804 external_mini_cluster.cc:1428] Setting key ac0c5106119ea34e468a9d02762b594d
I20260619 12:11:56.574014 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.565792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:33007 (local address 127.0.201.62:46683)
0619 12:11:56.565843 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.565845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.566046 (+ 201us) server_negotiation.cc:407] Connection header received
0619 12:11:56.566709 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.566711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.566723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.566783 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:56.567723 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.568221 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.569208 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.569409 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.569490 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.571454 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:56.571455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:56.571456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:56.571463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:56.572545 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.572937 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.572938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.572939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.573039 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.573285 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.573286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.573286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.573425 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:56.573497 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.573837 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.573916 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:11:56.574383 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.565751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:33007)
0619 12:11:56.565923 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.565965 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.566594 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.566852 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.566856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.567428 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:56.567614 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.567618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.568310 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.568311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.569113 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.569114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.569504 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.569505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.569639 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.570044 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:56.570052 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:56.571363 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:56.572636 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:56.572642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:56.572644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:56.572853 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:56.573090 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:56.573091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:56.573091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:56.573216 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:56.573517 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:56.573524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:56.573687 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.573805 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.573974 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260619 12:11:56.574788 15003 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46683
I20260619 12:11:56.574882 15003 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:56.575044 15003 heartbeater.cc:511] Master 127.0.201.62:46683 requested a full tablet report, sending...
I20260619 12:11:56.575407 14544 ts_manager.cc:194] Registered new tserver with Master: de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481)
I20260619 12:11:56.575789 14544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:33007
I20260619 12:11:56.584970 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:11:56.591954 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.586229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.0.201.62:46683)
0619 12:11:56.586292 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.586294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.586410 (+ 116us) server_negotiation.cc:407] Connection header received
0619 12:11:56.586454 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.586454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.586464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.586535 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:56.587155 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.587693 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.588410 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.588612 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.588694 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.589716 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:56.589717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:56.589718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:56.589725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:56.590820 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.591158 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.591159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.591160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.591235 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:56.591416 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:56.591417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:56.591417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:56.591556 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:56.591658 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.591783 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.591857 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:11:56.593281 14544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:11:56.594920 14544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:11:56.601102 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.597200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36003 (local address 127.0.0.1:50398)
0619 12:11:56.597654 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.597663 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.597710 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.597988 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.597990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.597992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:56.598042 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.598043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.599238 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.599239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.600126 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.600128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.600794 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.600795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.600911 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.600924 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.600984 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.601034 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":68,"threads_started":1}
I20260619 12:11:56.601123 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.597318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50398 (local address 127.0.201.1:36003)
0619 12:11:56.597509 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.597513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.597773 (+ 260us) server_negotiation.cc:407] Connection header received
0619 12:11:56.597818 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.597820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.597835 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.597901 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:56.598134 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.599121 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.600206 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.600689 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.600791 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.600860 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.601011 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.601050 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260619 12:11:56.601398 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.596994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44593 (local address 127.0.0.1:45514)
0619 12:11:56.597277 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.597318 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.597386 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.597572 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.597576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.597578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:56.597638 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.597639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.598937 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.598937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.600159 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.600161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.601151 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.601152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.601229 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.601234 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.601285 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.601321 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":29,"thread_start_us":44,"threads_started":1}
I20260619 12:11:56.601647 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.597157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.0.201.2:44593)
0619 12:11:56.597414 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.597417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.597427 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:11:56.597465 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.597467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.597482 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.597558 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:56.597724 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.598839 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.600249 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.600994 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.601064 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.601116 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.601506 (+ 390us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.601550 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":73,"threads_started":1}
I20260619 12:11:56.601886 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.597106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.0.201.3:44481)
0619 12:11:56.597340 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.597345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.597490 (+ 145us) server_negotiation.cc:407] Connection header received
0619 12:11:56.597535 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.597538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.597560 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.597642 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:56.597758 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.598658 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.600248 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.601006 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.601064 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.601115 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.601780 (+ 665us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.601826 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260619 12:11:56.601841 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 2a591bc8525c41a881e425877a96d004 (DEFAULT_TABLE table=test-table [id=b1f89ee522644816ae71ed70775d259c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:56.602066 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.596995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:44481 (local address 127.0.0.1:52328)
0619 12:11:56.597315 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.597325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.597419 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.597656 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.597658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.597660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:56.597705 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.597707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.598756 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.598757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.600147 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.600150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.601432 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.601434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.601567 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.601570 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.601623 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.601662 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":34,"thread_start_us":80,"threads_started":1}
I20260619 12:11:56.602187 14669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a591bc8525c41a881e425877a96d004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:56.602332 14803 tablet_service.cc:1467] Processing CreateTablet for tablet 2a591bc8525c41a881e425877a96d004 (DEFAULT_TABLE table=test-table [id=b1f89ee522644816ae71ed70775d259c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:56.602651 14803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a591bc8525c41a881e425877a96d004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:56.604732 15020 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap starting.
I20260619 12:11:56.605249 15021 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Bootstrap starting.
I20260619 12:11:56.605406 14937 tablet_service.cc:1467] Processing CreateTablet for tablet 2a591bc8525c41a881e425877a96d004 (DEFAULT_TABLE table=test-table [id=b1f89ee522644816ae71ed70775d259c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:56.605551 15020 tablet_bootstrap.cc:654] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:56.605741 14937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a591bc8525c41a881e425877a96d004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:56.605872 15021 tablet_bootstrap.cc:654] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:56.606119 15020 log.cc:824] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:56.606457 15021 log.cc:824] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:56.606912 15020 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: No bootstrap required, opened a new log
I20260619 12:11:56.606993 15020 ts_tablet_manager.cc:1397] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:11:56.607160 15021 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: No bootstrap required, opened a new log
I20260619 12:11:56.607254 15021 ts_tablet_manager.cc:1397] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:11:56.608906 15024 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Bootstrap starting.
I20260619 12:11:56.608882 15021 raft_consensus.cc:348] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.609058 15021 raft_consensus.cc:374] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:56.609093 15021 raft_consensus.cc:729] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f90cda8f49a472388a42c2e9a36cced, State: Initialized, Role: FOLLOWER
I20260619 12:11:56.609027 15020 raft_consensus.cc:348] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.609184 15020 raft_consensus.cc:374] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:56.609225 15020 raft_consensus.cc:729] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b31ed224e74f2fac1396f7665b5b8f, State: Initialized, Role: FOLLOWER
I20260619 12:11:56.609222 15021 consensus_queue.cc:260] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.609350 15020 consensus_queue.cc:260] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.609520 15021 ts_tablet_manager.cc:1428] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:11:56.609593 14869 heartbeater.cc:503] Master 127.0.201.62:46683 was elected leader, sending a full tablet report...
I20260619 12:11:56.609659 15024 tablet_bootstrap.cc:654] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:56.609915 15020 ts_tablet_manager.cc:1428] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:11:56.610033 14735 heartbeater.cc:503] Master 127.0.201.62:46683 was elected leader, sending a full tablet report...
I20260619 12:11:56.610180 15024 log.cc:824] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:56.610862 15024 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: No bootstrap required, opened a new log
I20260619 12:11:56.610939 15024 ts_tablet_manager.cc:1397] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:11:56.612215 15024 raft_consensus.cc:348] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.612355 15024 raft_consensus.cc:374] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:56.612399 15024 raft_consensus.cc:729] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9ddd93604345dba2de5001ed497449, State: Initialized, Role: FOLLOWER
I20260619 12:11:56.612511 15024 consensus_queue.cc:260] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.612810 15024 ts_tablet_manager.cc:1428] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:11:56.612818 15003 heartbeater.cc:503] Master 127.0.201.62:46683 was elected leader, sending a full tablet report...
I20260619 12:11:56.629817 15028 raft_consensus.cc:482] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:11:56.629938 15028 raft_consensus.cc:504] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.630270 15028 leader_election.cc:288] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593), 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003)
I20260619 12:11:56.633222 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.630471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:38639 (local address 127.0.201.2:44593)
0619 12:11:56.630512 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.630514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.630522 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:11:56.630548 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.630549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.630560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.630621 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:56.630787 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.631363 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.632479 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.632857 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.632913 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.632943 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.633115 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.633152 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:11:56.633253 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.630387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44593 (local address 127.0.201.3:38639)
0619 12:11:56.630430 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.630443 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.630484 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.630642 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.630643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.630644 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:56.630716 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.630718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.631443 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.631444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.632399 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.632402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.632939 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.632940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.633038 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.633044 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.633092 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.633148 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":27}
I20260619 12:11:56.633245 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.630504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:55935 (local address 127.0.201.1:36003)
0619 12:11:56.630544 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.630545 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.630618 (+ 73us) server_negotiation.cc:407] Connection header received
0619 12:11:56.630673 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.630674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.630684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.630738 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:56.630917 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.631422 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.632525 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.632873 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.632956 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.632965 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.633144 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.633181 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:11:56.633275 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.630400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36003 (local address 127.0.201.3:55935)
0619 12:11:56.630565 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.630575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:56.630625 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:56.630774 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:56.630774 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:56.630776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:56.630838 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.630840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.631512 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.631512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.632410 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:56.632413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.632976 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.632977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:56.633052 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.633055 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:11:56.633127 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:11:56.633164 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260619 12:11:56.633584 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a591bc8525c41a881e425877a96d004" candidate_uuid: "de9ddd93604345dba2de5001ed497449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b31ed224e74f2fac1396f7665b5b8f" is_pre_election: true
I20260619 12:11:56.633570 14823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a591bc8525c41a881e425877a96d004" candidate_uuid: "de9ddd93604345dba2de5001ed497449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f90cda8f49a472388a42c2e9a36cced" is_pre_election: true
I20260619 12:11:56.633718 14689 raft_consensus.cc:2454] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de9ddd93604345dba2de5001ed497449 in term 0.
I20260619 12:11:56.633733 14823 raft_consensus.cc:2454] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de9ddd93604345dba2de5001ed497449 in term 0.
I20260619 12:11:56.633926 14891 leader_election.cc:302] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f90cda8f49a472388a42c2e9a36cced, de9ddd93604345dba2de5001ed497449; no voters:
I20260619 12:11:56.634070 15028 raft_consensus.cc:2781] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:11:56.634121 15028 raft_consensus.cc:482] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:11:56.634153 15028 raft_consensus.cc:3037] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:56.634788 15028 raft_consensus.cc:504] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.634918 15028 leader_election.cc:288] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [CANDIDATE]: Term 1 election: Requested vote from peers 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593), 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003)
I20260619 12:11:56.635066 14823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a591bc8525c41a881e425877a96d004" candidate_uuid: "de9ddd93604345dba2de5001ed497449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f90cda8f49a472388a42c2e9a36cced"
I20260619 12:11:56.635102 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a591bc8525c41a881e425877a96d004" candidate_uuid: "de9ddd93604345dba2de5001ed497449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b31ed224e74f2fac1396f7665b5b8f"
I20260619 12:11:56.635149 14823 raft_consensus.cc:3037] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:56.635159 14689 raft_consensus.cc:3037] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:56.636047 14823 raft_consensus.cc:2454] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de9ddd93604345dba2de5001ed497449 in term 1.
I20260619 12:11:56.636076 14689 raft_consensus.cc:2454] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de9ddd93604345dba2de5001ed497449 in term 1.
I20260619 12:11:56.636209 14890 leader_election.cc:302] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f, de9ddd93604345dba2de5001ed497449; no voters:
I20260619 12:11:56.636292 15028 raft_consensus.cc:2781] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:56.636420 15028 raft_consensus.cc:686] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 1 LEADER]: Becoming Leader. State: Replica: de9ddd93604345dba2de5001ed497449, State: Running, Role: LEADER
I20260619 12:11:56.636499 15028 consensus_queue.cc:237] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:56.637239 14544 catalog_manager.cc:5495] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 reported cstate change: term changed from 0 to 1, leader changed from <none> to de9ddd93604345dba2de5001ed497449 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "de9ddd93604345dba2de5001ed497449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } health_report { overall_health: UNKNOWN } } }
I20260619 12:11:56.645052 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.643083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.0.201.3:44481)
0619 12:11:56.643119 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:56.643120 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:56.643131 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:11:56.643241 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:56.643242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:56.643258 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:56.643321 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:11:56.643423 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.643845 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.644517 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:56.644680 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:56.644727 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:56.644833 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:11:56.644908 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:11:56.644957 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:56.644997 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 12:11:56.645010 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:11:56.646509 14689 raft_consensus.cc:1261] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Refusing update from remote peer de9ddd93604345dba2de5001ed497449: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:56.646524 14823 raft_consensus.cc:1261] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Refusing update from remote peer de9ddd93604345dba2de5001ed497449: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:56.646793 15030 consensus_queue.cc:1035] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:56.646916 15028 consensus_queue.cc:1035] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:56.649134 15033 mvcc.cc:205] Tried to move back new op lower bound from 7298544093782122496 to 7298544093743800320. Current Snapshot: MvccSnapshot[applied={T|T < 7298544093782122496}]
I20260619 12:11:56.649698 15035 mvcc.cc:205] Tried to move back new op lower bound from 7298544093782122496 to 7298544093743800320. Current Snapshot: MvccSnapshot[applied={T|T < 7298544093782122496}]
I20260619 12:11:56.650861 15034 mvcc.cc:205] Tried to move back new op lower bound from 7298544093782122496 to 7298544093743800320. Current Snapshot: MvccSnapshot[applied={T|T < 7298544093782122496 or (T in {7298544093782122496})}]
I20260619 12:11:56.651175 14544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:56.651293 14544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:56.651948 14544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Sending DeleteTablet for 3 replicas of tablet 2a591bc8525c41a881e425877a96d004
I20260619 12:11:56.652266 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a591bc8525c41a881e425877a96d004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:56 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:50398
I20260619 12:11:56.652267 14937 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a591bc8525c41a881e425877a96d004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:56 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:52328
I20260619 12:11:56.652275 14803 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a591bc8525c41a881e425877a96d004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:56 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:45514
I20260619 12:11:56.652307 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14606
I20260619 12:11:56.655323 15043 tablet_replica.cc:330] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: stopping tablet replica
I20260619 12:11:56.655424 15043 raft_consensus.cc:2229] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:11:56.655529 15043 raft_consensus.cc:2258] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:56.655674 15042 tablet_replica.cc:330] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: stopping tablet replica
I20260619 12:11:56.655756 15042 raft_consensus.cc:2229] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:11:56.655838 15042 raft_consensus.cc:2258] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:56.656240 15043 ts_tablet_manager.cc:1905] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:56.656262 15042 ts_tablet_manager.cc:1905] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:56.657995 15043 ts_tablet_manager.cc:1918] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:56.658069 15043 log.cc:1196] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/2a591bc8525c41a881e425877a96d004
I20260619 12:11:56.658339 15043 ts_tablet_manager.cc:1939] T 2a591bc8525c41a881e425877a96d004 P 8f90cda8f49a472388a42c2e9a36cced: Deleting consensus metadata
I20260619 12:11:56.658440 15042 ts_tablet_manager.cc:1918] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:56.658574 15042 log.cc:1196] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/2a591bc8525c41a881e425877a96d004
I20260619 12:11:56.658728 14529 catalog_manager.cc:4841] TS 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): tablet 2a591bc8525c41a881e425877a96d004 (table test-table [id=b1f89ee522644816ae71ed70775d259c]) successfully deleted
W20260619 12:11:56.658730 14528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:36003 (error 108)
I20260619 12:11:56.658965 15042 ts_tablet_manager.cc:1939] T 2a591bc8525c41a881e425877a96d004 P de9ddd93604345dba2de5001ed497449: Deleting consensus metadata
I20260619 12:11:56.659124 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14740
I20260619 12:11:56.660550 14528 catalog_manager.cc:4841] TS de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): tablet 2a591bc8525c41a881e425877a96d004 (table test-table [id=b1f89ee522644816ae71ed70775d259c]) successfully deleted
I20260619 12:11:56.663581 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:56.663448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36003 (local address 127.0.0.1:50406)
0619 12:11:56.663510 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:56.663550 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:11:56.663723 14528 catalog_manager.cc:4584] TS 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a591bc8525c41a881e425877a96d004: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36003: connect: Connection refused (error 111)
I20260619 12:11:56.665577 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14874
I20260619 12:11:56.670689 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 14512
I20260619 12:11:56.675719 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46683
--webserver_interface=127.0.201.62
--webserver_port=46689
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:56.754277 15045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:56.754474 15045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:56.754496 15045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:56.755687 15045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:56.755726 15045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:56.755740 15045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:56.755753 15045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:56.755765 15045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:56.757179 15045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=46689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:56.757427 15045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:56.757651 15045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:56.765944 15054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.765924 15051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.765916 15052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.766049 15045 server_base.cc:1034] running on GCE node
I20260619 12:11:56.766320 15045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:56.766634 15045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:56.767781 15045 hybrid_clock.cc:648] HybridClock initialized: now 1781871116767766 us; error 47 us; skew 500 ppm
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:56.770198 15045 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:56.770627 15045 webserver.cc:460] Webserver started at http://127.0.201.62:46689/ using document root <none> and password file <none>
I20260619 12:11:56.770807 15045 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:56.770855 15045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:56.772168 15045 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:56.772753 15061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.772976 15045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:56.773075 15045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "b15f8925ffcc4226aa7b9adcf2f5803b"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "1724013940101a913fe94685e1822d1d"
server_key_iv: "7513b932d9bc24f4d247df2da4ff0e5c"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.773415 15045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:56.817821 15045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:56.820255 15045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:56.820370 15045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:56.824309 15045 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:46683
I20260619 12:11:56.824381 15113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:46683 every 8 connection(s)
I20260619 12:11:56.824751 15045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:56.826241 15114 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:11:56.826825 15114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap starting.
I20260619 12:11:56.830529 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15045
I20260619 12:11:56.830813 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:36003
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:46683
--webserver_port=43265
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:56.835045 15114 log.cc:824] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:56.836834 15114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:11:56.837134 15114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap complete.
I20260619 12:11:56.839372 15114 raft_consensus.cc:348] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.839620 15114 raft_consensus.cc:729] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15f8925ffcc4226aa7b9adcf2f5803b, State: Initialized, Role: FOLLOWER
I20260619 12:11:56.839794 15114 consensus_queue.cc:260] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.839871 15114 raft_consensus.cc:388] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:56.839926 15114 raft_consensus.cc:482] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:56.839975 15114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:11:56.840930 15114 raft_consensus.cc:504] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.841054 15114 leader_election.cc:302] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b15f8925ffcc4226aa7b9adcf2f5803b; no voters:
I20260619 12:11:56.841262 15114 leader_election.cc:288] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:11:56.841292 15118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:11:56.841416 15118 raft_consensus.cc:686] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 LEADER]: Becoming Leader. State: Replica: b15f8925ffcc4226aa7b9adcf2f5803b, State: Running, Role: LEADER
I20260619 12:11:56.841531 15114 sys_catalog.cc:564] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:56.841553 15118 consensus_queue.cc:237] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:56.841884 15119 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } } }
I20260619 12:11:56.841951 15119 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:56.842159 15120 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b15f8925ffcc4226aa7b9adcf2f5803b. Latest consensus state: current_term: 2 leader_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } } }
I20260619 12:11:56.842278 15120 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:56.843442 15134 catalog_manager.cc:1259] Loaded cluster ID: e8bed7320c7246d398aeb96abe673b1f
I20260619 12:11:56.843490 15134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: loading cluster ID for follower catalog manager: success
W20260619 12:11:56.843667 15134 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260619 12:11:56.843706 15134 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260619 12:11:56.843737 15123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:56.844075 15123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:56.844141 15123 catalog_manager.cc:1259] Loaded cluster ID: e8bed7320c7246d398aeb96abe673b1f
I20260619 12:11:56.844162 15123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:11:56.844314 15123 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 12:11:56.849020 15123 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 12:11:56.851439 15123 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:11:56.851612 15123 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Loaded TSK: 0
I20260619 12:11:56.851827 15123 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:11:56.913260 15116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:56.913477 15116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:56.913516 15116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:56.914634 15116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:56.914700 15116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:56.914745 15116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:11:56.916287 15116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:36003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=43265
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:56.916543 15116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:56.916764 15116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:56.921622 15141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.921692 15144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:56.921629 15142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:56.922135 15116 server_base.cc:1034] running on GCE node
I20260619 12:11:56.922325 15116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:56.922570 15116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:56.923748 15116 hybrid_clock.cc:648] HybridClock initialized: now 1781871116923737 us; error 39 us; skew 500 ppm
Jun 19 12:11:56 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:56.931740 15116 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:11:56.932216 15116 webserver.cc:460] Webserver started at http://127.0.201.1:43265/ using document root <none> and password file <none>
I20260619 12:11:56.932440 15116 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:56.932492 15116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:56.933842 15116 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:56.934513 15151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:56.934664 15116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:56.934764 15116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "39b31ed224e74f2fac1396f7665b5b8f"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "555007d0bf3cabce02fdd45f5b91940d"
server_key_iv: "c911a07d85c92125ebb61e9c644495bf"
server_key_version: "encryptionkey@0"
I20260619 12:11:56.935081 15116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:56.994644 15116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:56.996971 15116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:56.997148 15116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:56.997381 15116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:56.997813 15158 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:11:56.999184 15116 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:11:56.999264 15116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 12:11:56.999310 15116 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:11:56.999908 15116 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:11:56.999962 15116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 12:11:57.000027 15158 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap starting.
I20260619 12:11:57.006358 15116 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:36003
I20260619 12:11:57.006443 15265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:36003 every 8 connection(s)
I20260619 12:11:57.006724 15116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:11:57.010919 15158 log.cc:824] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:57.013437 15158 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:11:57.013751 15158 tablet_bootstrap.cc:492] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap complete.
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:57.014760 15158 ts_tablet_manager.cc:1397] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260619 12:11:57.015746 15158 raft_consensus.cc:348] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.015971 15158 raft_consensus.cc:729] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b31ed224e74f2fac1396f7665b5b8f, State: Initialized, Role: FOLLOWER
I20260619 12:11:57.016095 15158 consensus_queue.cc:260] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.016247 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15116
I20260619 12:11:57.016399 15158 ts_tablet_manager.cc:1428] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260619 12:11:57.016526 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:44593
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:46683
--webserver_port=33637
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:57.018632 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.007530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:38851 (local address 127.0.201.62:46683)
0619 12:11:57.007708 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.007720 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.007749 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:11:57.008564 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.008569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.008613 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.008712 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:57.009658 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.010505 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.011918 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.012351 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.012438 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.014943 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:57.014949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:57.014950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:57.014964 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:57.016957 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.017379 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.017382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.017382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.017479 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.017727 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.017728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.017729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.017948 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:57.018033 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.018322 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.018446 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260619 12:11:57.018631 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.007415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:38851)
0619 12:11:57.007579 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.007621 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.008349 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.008791 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.008796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.009208 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:57.009546 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.009554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.010623 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.010624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.011823 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.011826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.012465 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.012466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.012630 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.013153 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:57.013162 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:57.014759 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:57.017053 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.017059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.017061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.017279 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.017550 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.017550 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.017551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.017657 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.018027 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:57.018033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:57.018165 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.018270 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.018443 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260619 12:11:57.019652 15266 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46683
I20260619 12:11:57.019749 15266 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:57.019905 15266 heartbeater.cc:511] Master 127.0.201.62:46683 requested a full tablet report, sending...
I20260619 12:11:57.020558 15078 ts_manager.cc:194] Registered new tserver with Master: 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003)
I20260619 12:11:57.021588 15078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:38851
I20260619 12:11:57.022356 15266 heartbeater.cc:503] Master 127.0.201.62:46683 was elected leader, sending a full tablet report...
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.1@KRBTEST.COM
I20260619 12:11:57.029316 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.021123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36003 (local address 127.0.0.1:50412)
0619 12:11:57.021340 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.021386 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.021464 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.021749 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.021755 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.022120 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:57.022180 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.022183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.023615 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.023616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.024158 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.024160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.024824 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.024825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.024872 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.025308 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:57.025312 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:57.026487 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:57.028330 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.028335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.028337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.028513 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.028734 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.028734 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.028735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.028784 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.029056 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:57.029061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:57.029098 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.029156 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.029236 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260619 12:11:57.029398 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.021183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.0.201.1:36003)
0619 12:11:57.021321 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.021325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.021460 (+ 135us) server_negotiation.cc:407] Connection header received
0619 12:11:57.021552 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.021554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.021569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.021649 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:57.022282 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.023537 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.024240 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.024746 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.024827 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.026569 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:57.026577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:57.026578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:57.026585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:57.028248 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.028609 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.028612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.028612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.028673 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.028839 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.028840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.028840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.028995 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:57.029071 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.029211 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.029299 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260619 12:11:57.029691 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a591bc8525c41a881e425877a96d004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:56 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:50412
I20260619 12:11:57.030230 15275 tablet_replica.cc:330] stopping tablet replica
I20260619 12:11:57.030324 15275 raft_consensus.cc:2229] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:11:57.030385 15275 raft_consensus.cc:2258] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:57.030630 15275 ts_tablet_manager.cc:1905] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:57.032618 15275 ts_tablet_manager.cc:1918] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:57.032676 15275 log.cc:1196] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/2a591bc8525c41a881e425877a96d004
I20260619 12:11:57.032887 15275 ts_tablet_manager.cc:1939] T 2a591bc8525c41a881e425877a96d004 P 39b31ed224e74f2fac1396f7665b5b8f: Deleting consensus metadata
I20260619 12:11:57.033337 15062 catalog_manager.cc:4841] TS 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003): tablet 2a591bc8525c41a881e425877a96d004 (table test-table [id=b1f89ee522644816ae71ed70775d259c]) successfully deleted
W20260619 12:11:57.098448 15272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:57.098667 15272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:57.098704 15272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:57.099833 15272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:57.099903 15272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:57.099951 15272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:11:57.101342 15272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:44593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=33637
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:57.101589 15272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:57.101809 15272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:57.106509 15283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.106590 15280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.106593 15281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.106647 15272 server_base.cc:1034] running on GCE node
I20260619 12:11:57.106940 15272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:57.107162 15272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:57.108340 15272 hybrid_clock.cc:648] HybridClock initialized: now 1781871117108323 us; error 33 us; skew 500 ppm
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:57.110630 15272 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:11:57.111084 15272 webserver.cc:460] Webserver started at http://127.0.201.2:33637/ using document root <none> and password file <none>
I20260619 12:11:57.111351 15272 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:57.111431 15272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:57.112661 15272 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:57.113230 15290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.113392 15272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.113484 15272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "8f90cda8f49a472388a42c2e9a36cced"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "98167e40c8cb7ff0462f29103c5d2a0e"
server_key_iv: "6d1380b69b314b63968ca0980de6cad0"
server_key_version: "encryptionkey@0"
I20260619 12:11:57.113797 15272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:57.123358 15272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:57.124975 15272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:57.125126 15272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:57.125344 15272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:57.125612 15272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:57.125661 15272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.125699 15272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:57.125725 15272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.131395 15272 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44593
I20260619 12:11:57.131438 15403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44593 every 8 connection(s)
I20260619 12:11:57.131745 15272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:57.140307 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.132386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:55533 (local address 127.0.201.62:46683)
0619 12:11:57.132448 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.132450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.132657 (+ 207us) server_negotiation.cc:407] Connection header received
0619 12:11:57.133091 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.133093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.133102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.133155 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:57.133996 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.134469 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.135471 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.135663 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.135714 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.137728 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:57.137730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:57.137730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:57.137736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:57.139083 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.139574 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.139575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.139575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.139624 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.139836 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.139836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.139836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.139926 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:57.139982 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.140183 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.140246 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:11:57.140514 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.132366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:55533)
0619 12:11:57.132530 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.132568 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.133002 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.133202 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.133205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.133734 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:57.133912 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.133916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.134549 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.134550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.135401 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.135404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.135735 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.135736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.135833 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.136227 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:57.136235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:57.137643 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:57.139174 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.139179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.139181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.139505 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.139683 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.139683 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.139684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.139783 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.139978 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:57.139982 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:57.140082 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.140167 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.140273 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260619 12:11:57.141060 15404 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46683
I20260619 12:11:57.141162 15404 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:57.141364 15404 heartbeater.cc:511] Master 127.0.201.62:46683 requested a full tablet report, sending...
I20260619 12:11:57.141537 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15272
I20260619 12:11:57.141789 15078 ts_manager.cc:194] Registered new tserver with Master: 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
I20260619 12:11:57.141858 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:44481
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:46683
--webserver_port=38519
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:57.142256 15078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:55533
W20260619 12:11:57.223488 15407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:57.223717 15407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:57.223755 15407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:57.224897 15407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:57.224961 15407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:57.225006 15407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:11:57.226414 15407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:44481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=38519
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:57.226671 15407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:57.226881 15407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:57.231640 15412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.231640 15415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.231637 15413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.231983 15407 server_base.cc:1034] running on GCE node
I20260619 12:11:57.232219 15407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:57.232441 15407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:57.233596 15407 hybrid_clock.cc:648] HybridClock initialized: now 1781871117233593 us; error 35 us; skew 500 ppm
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:57.236145 15407 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:11:57.236497 15407 webserver.cc:460] Webserver started at http://127.0.201.3:38519/ using document root <none> and password file <none>
I20260619 12:11:57.236708 15407 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:57.236774 15407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:57.238076 15407 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:57.238662 15422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.238829 15407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:11:57.238894 15407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "de9ddd93604345dba2de5001ed497449"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "86267b2c3bb489646ca0b7285c017367"
server_key_iv: "3b74a92ca976bfd9fa6622b3939da621"
server_key_version: "encryptionkey@0"
I20260619 12:11:57.239251 15407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:57.258849 15407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:57.261128 15407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:57.261272 15407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:57.261512 15407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:57.261821 15407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:57.261874 15407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.261914 15407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:57.261941 15407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.267455 15407 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:44481
I20260619 12:11:57.267534 15535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:44481 every 8 connection(s)
I20260619 12:11:57.267858 15407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:11:57.276530 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.268523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:50823 (local address 127.0.201.62:46683)
0619 12:11:57.268590 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.268592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.268834 (+ 242us) server_negotiation.cc:407] Connection header received
0619 12:11:57.269332 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.269334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.269371 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.269426 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:57.270332 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.270833 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.271901 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.272105 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.272192 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.274185 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:57.274187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:57.274187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:57.274193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:57.275479 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.275806 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.275806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.275807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.275858 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.276060 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.276060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.276060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.276151 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:57.276199 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.276412 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.276469 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:11:57.276703 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.268519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:50823)
0619 12:11:57.268681 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.268739 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.269224 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.269493 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.269497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.270045 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:57.270227 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.270233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.270927 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.270929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.271789 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.271792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.272199 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.272200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.272346 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.272820 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:57.272828 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:57.274101 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:57.275541 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.275546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.275548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.275745 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.275929 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:57.275929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:57.275930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:57.276019 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:57.276203 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:57.276207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:57.276325 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.276400 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.276515 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260619 12:11:57.277215 15536 heartbeater.cc:347] Connected to a master server at 127.0.201.62:46683
I20260619 12:11:57.277315 15536 heartbeater.cc:464] Registering TS with master...
I20260619 12:11:57.277328 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15407
I20260619 12:11:57.277506 15536 heartbeater.cc:511] Master 127.0.201.62:46683 requested a full tablet report, sending...
I20260619 12:11:57.277948 15078 ts_manager.cc:194] Registered new tserver with Master: de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481)
I20260619 12:11:57.278308 15078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:50823
I20260619 12:11:57.283506 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.277699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.0.201.62:46683)
0619 12:11:57.277737 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.277739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.278414 (+ 675us) server_negotiation.cc:407] Connection header received
0619 12:11:57.278434 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.278435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.278445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.278522 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:57.279237 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.279668 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.280344 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.280510 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.280555 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.281506 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:57.281506 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:57.281507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:57.281512 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:57.282584 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.282871 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.282872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.282872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.282916 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:57.283045 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:57.283045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:57.283045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:57.283127 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:57.283303 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.283348 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.283421 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:11:57.284188 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:11:57.286583 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.284473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32784 (local address 127.0.201.62:46683)
0619 12:11:57.284533 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.284535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.284579 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:11:57.284631 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.284632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.284655 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.284723 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:11:57.284787 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.285226 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.285998 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.286160 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.286219 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.286380 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:11:57.286457 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:11:57.286520 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.286533 (+ 13us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.286545 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:11:57.286867 15078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:32784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:11:57.288492 15078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:11:57.290393 15200 tablet_service.cc:1467] Processing CreateTablet for tablet 72e3da61c41b4eb183a8dab899693d6e (DEFAULT_TABLE table=test-table [id=6f1a663bd3e047ebaaa4a7f159e7c0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:57.290596 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e3da61c41b4eb183a8dab899693d6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:57.292045 15158 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap starting.
I20260619 12:11:57.292379 15158 tablet_bootstrap.cc:654] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:57.292927 15158 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: No bootstrap required, opened a new log
I20260619 12:11:57.293010 15158 ts_tablet_manager.cc:1397] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:57.293401 15158 raft_consensus.cc:348] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.293459 15158 raft_consensus.cc:374] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:57.293484 15158 raft_consensus.cc:729] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b31ed224e74f2fac1396f7665b5b8f, State: Initialized, Role: FOLLOWER
I20260619 12:11:57.293545 15158 consensus_queue.cc:260] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.293633 15158 ts_tablet_manager.cc:1428] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:57.293941 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.290073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:44481 (local address 127.0.0.1:52348)
0619 12:11:57.290119 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.290130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.290164 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.290522 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.290523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.290524 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:57.290563 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.290564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.291722 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.291723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.292735 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.292737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.293677 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.293678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.293747 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.293765 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.293819 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.293871 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:11:57.294248 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.290212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.0.201.3:44481)
0619 12:11:57.290367 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.290371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.290380 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:11:57.290411 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.290413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.290427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.290503 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:57.290613 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.291630 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.292826 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.293529 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.293590 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.293644 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.294129 (+ 485us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.294170 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260619 12:11:57.294852 15470 tablet_service.cc:1467] Processing CreateTablet for tablet 72e3da61c41b4eb183a8dab899693d6e (DEFAULT_TABLE table=test-table [id=6f1a663bd3e047ebaaa4a7f159e7c0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:57.295146 15470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e3da61c41b4eb183a8dab899693d6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:57.295631 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.290299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.0.201.2:44593)
0619 12:11:57.292122 (+ 1823us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.292127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.292137 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:11:57.292174 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.292176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.292192 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.292291 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:57.292394 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.293463 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.294721 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.295334 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.295406 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.295466 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.295546 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.295574 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1785,"thread_start_us":83,"threads_started":1}
I20260619 12:11:57.295646 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.290122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44593 (local address 127.0.0.1:45518)
0619 12:11:57.290272 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.290283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.290332 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.292287 (+ 1955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.292288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.292289 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:57.292336 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.292338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.293610 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.293611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.294644 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.294646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.295407 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.295408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.295470 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.295475 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.295538 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.295575 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":44,"threads_started":1}
I20260619 12:11:57.296248 15338 tablet_service.cc:1467] Processing CreateTablet for tablet 72e3da61c41b4eb183a8dab899693d6e (DEFAULT_TABLE table=test-table [id=6f1a663bd3e047ebaaa4a7f159e7c0e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:11:57.296530 15338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e3da61c41b4eb183a8dab899693d6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:11:57.298000 15545 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Bootstrap starting.
I20260619 12:11:57.298671 15545 tablet_bootstrap.cc:654] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:57.299260 15545 log.cc:824] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:57.299470 15546 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Bootstrap starting.
I20260619 12:11:57.299918 15545 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: No bootstrap required, opened a new log
I20260619 12:11:57.299969 15545 ts_tablet_manager.cc:1397] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:11:57.300050 15546 tablet_bootstrap.cc:654] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Neither blocks nor log segments found. Creating new log.
I20260619 12:11:57.300535 15546 log.cc:824] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:57.301327 15546 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: No bootstrap required, opened a new log
I20260619 12:11:57.301414 15546 ts_tablet_manager.cc:1397] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:11:57.301520 15545 raft_consensus.cc:348] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.301656 15545 raft_consensus.cc:374] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:57.301679 15545 raft_consensus.cc:729] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9ddd93604345dba2de5001ed497449, State: Initialized, Role: FOLLOWER
I20260619 12:11:57.301805 15545 consensus_queue.cc:260] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.302052 15545 ts_tablet_manager.cc:1428] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:11:57.302076 15536 heartbeater.cc:503] Master 127.0.201.62:46683 was elected leader, sending a full tablet report...
I20260619 12:11:57.303349 15546 raft_consensus.cc:348] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.303495 15546 raft_consensus.cc:374] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:11:57.303539 15546 raft_consensus.cc:729] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f90cda8f49a472388a42c2e9a36cced, State: Initialized, Role: FOLLOWER
I20260619 12:11:57.303656 15546 consensus_queue.cc:260] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.303889 15546 ts_tablet_manager.cc:1428] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:11:57.303923 15404 heartbeater.cc:503] Master 127.0.201.62:46683 was elected leader, sending a full tablet report...
W20260619 12:11:57.382249 15405 tablet.cc:2369] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:11:57.385962 15271 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:11:57.386070 15271 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.386402 15271 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
I20260619 12:11:57.390583 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.386539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:44481 (local address 127.0.201.1:36223)
0619 12:11:57.386645 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.386660 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.386735 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.386948 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.386949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.386950 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:57.387037 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.387039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.388124 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.388125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.389424 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.389427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.390268 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.390269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.390355 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.390367 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.390434 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.390484 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":84}
I20260619 12:11:57.390588 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.386759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47967 (local address 127.0.201.2:44593)
0619 12:11:57.386815 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.386817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.386962 (+ 145us) server_negotiation.cc:407] Connection header received
0619 12:11:57.387019 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.387020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.387033 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.387086 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:57.387284 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.388048 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.389528 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.390176 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.390259 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.390270 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.390458 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.390510 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:11:57.390588 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.386600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:36223 (local address 127.0.201.3:44481)
0619 12:11:57.386667 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.386669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.386707 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:11:57.386846 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.386848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.386861 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.386917 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:11:57.387147 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.388034 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.389528 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.390176 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.390259 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.390269 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.390458 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.390502 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:11:57.390595 15551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.386668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44593 (local address 127.0.201.1:47967)
0619 12:11:57.386855 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.386866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.386958 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.387108 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:57.387108 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:57.387110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:11:57.387213 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.387214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.388124 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.388125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.389424 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:57.389427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.390282 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.390283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:57.390343 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.390368 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 12:11:57.390434 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 12:11:57.390484 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":53,"thread_start_us":109,"threads_started":1}
I20260619 12:11:57.390981 15490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e3da61c41b4eb183a8dab899693d6e" candidate_uuid: "39b31ed224e74f2fac1396f7665b5b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9ddd93604345dba2de5001ed497449" is_pre_election: true
I20260619 12:11:57.390992 15358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e3da61c41b4eb183a8dab899693d6e" candidate_uuid: "39b31ed224e74f2fac1396f7665b5b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f90cda8f49a472388a42c2e9a36cced" is_pre_election: true
I20260619 12:11:57.391139 15490 raft_consensus.cc:2454] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39b31ed224e74f2fac1396f7665b5b8f in term 0.
I20260619 12:11:57.391178 15358 raft_consensus.cc:2454] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39b31ed224e74f2fac1396f7665b5b8f in term 0.
I20260619 12:11:57.391368 15152 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f, de9ddd93604345dba2de5001ed497449; no voters:
I20260619 12:11:57.391541 15271 raft_consensus.cc:2781] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:11:57.391587 15271 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:11:57.391601 15271 raft_consensus.cc:3037] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:57.392292 15271 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.392426 15271 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 1 election: Requested vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
I20260619 12:11:57.392612 15490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e3da61c41b4eb183a8dab899693d6e" candidate_uuid: "39b31ed224e74f2fac1396f7665b5b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9ddd93604345dba2de5001ed497449"
I20260619 12:11:57.392709 15358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e3da61c41b4eb183a8dab899693d6e" candidate_uuid: "39b31ed224e74f2fac1396f7665b5b8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f90cda8f49a472388a42c2e9a36cced"
I20260619 12:11:57.392706 15490 raft_consensus.cc:3037] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:57.392772 15358 raft_consensus.cc:3037] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:11:57.393476 15358 raft_consensus.cc:2454] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b31ed224e74f2fac1396f7665b5b8f in term 1.
I20260619 12:11:57.393507 15490 raft_consensus.cc:2454] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39b31ed224e74f2fac1396f7665b5b8f in term 1.
I20260619 12:11:57.393669 15152 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f, de9ddd93604345dba2de5001ed497449; no voters:
I20260619 12:11:57.393790 15271 raft_consensus.cc:2781] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:11:57.393947 15271 raft_consensus.cc:686] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 LEADER]: Becoming Leader. State: Replica: 39b31ed224e74f2fac1396f7665b5b8f, State: Running, Role: LEADER
I20260619 12:11:57.394037 15271 consensus_queue.cc:237] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.394816 15077 catalog_manager.cc:5495] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1). New cstate: current_term: 1 leader_uuid: "39b31ed224e74f2fac1396f7665b5b8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } health_report { overall_health: HEALTHY } } }
I20260619 12:11:57.411801 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.409278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.0.201.1:36003)
0619 12:11:57.409338 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:57.409340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:57.409413 (+ 73us) server_negotiation.cc:407] Connection header received
0619 12:11:57.409501 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:57.409502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:57.409519 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:57.409589 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:11:57.409727 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.410220 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.410961 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:57.411188 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:57.411292 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:57.411455 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:11:57.411567 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:11:57.411652 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:57.411696 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 12:11:57.411718 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:11:57.413293 15358 raft_consensus.cc:1261] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Refusing update from remote peer 39b31ed224e74f2fac1396f7665b5b8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:57.413321 15490 raft_consensus.cc:1261] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Refusing update from remote peer 39b31ed224e74f2fac1396f7665b5b8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:11:57.413611 15552 consensus_queue.cc:1035] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:57.413777 15271 consensus_queue.cc:1035] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:11:57.416029 15555 mvcc.cc:205] Tried to move back new op lower bound from 7298544096922398720 to 7298544096846569472. Current Snapshot: MvccSnapshot[applied={T|T < 7298544096922398720}]
I20260619 12:11:57.416103 15557 mvcc.cc:205] Tried to move back new op lower bound from 7298544096922398720 to 7298544096846569472. Current Snapshot: MvccSnapshot[applied={T|T < 7298544096922398720}]
I20260619 12:11:57.416111 15556 mvcc.cc:205] Tried to move back new op lower bound from 7298544096922398720 to 7298544096846569472. Current Snapshot: MvccSnapshot[applied={T|T < 7298544096922398720}]
I20260619 12:11:57.418097 15078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:32784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:57.418190 15078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:32784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:11:57.418915 15078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Sending DeleteTablet for 3 replicas of tablet 72e3da61c41b4eb183a8dab899693d6e
I20260619 12:11:57.419188 15470 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e3da61c41b4eb183a8dab899693d6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:57 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:52348
I20260619 12:11:57.419265 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e3da61c41b4eb183a8dab899693d6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:57 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:50412
I20260619 12:11:57.419266 15338 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e3da61c41b4eb183a8dab899693d6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:11:57 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:45518
I20260619 12:11:57.419355 15275 tablet_replica.cc:330] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: stopping tablet replica
I20260619 12:11:57.419417 15275 raft_consensus.cc:2229] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:11:57.419490 15275 raft_consensus.cc:2258] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:57.419641 15564 tablet_replica.cc:330] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: stopping tablet replica
I20260619 12:11:57.419698 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15116
I20260619 12:11:57.419739 15564 raft_consensus.cc:2229] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:11:57.420415 15564 raft_consensus.cc:2258] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:57.419669 15565 tablet_replica.cc:330] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: stopping tablet replica
I20260619 12:11:57.420565 15565 raft_consensus.cc:2229] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:11:57.420703 15565 raft_consensus.cc:2258] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:11:57.420861 15564 ts_tablet_manager.cc:1905] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:57.421135 15565 ts_tablet_manager.cc:1905] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:11:57.422698 15565 ts_tablet_manager.cc:1918] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:57.422700 15564 ts_tablet_manager.cc:1918] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:11:57.422786 15564 log.cc:1196] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/72e3da61c41b4eb183a8dab899693d6e
I20260619 12:11:57.422791 15565 log.cc:1196] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/72e3da61c41b4eb183a8dab899693d6e
I20260619 12:11:57.423074 15565 ts_tablet_manager.cc:1939] T 72e3da61c41b4eb183a8dab899693d6e P 8f90cda8f49a472388a42c2e9a36cced: Deleting consensus metadata
I20260619 12:11:57.423077 15564 ts_tablet_manager.cc:1939] T 72e3da61c41b4eb183a8dab899693d6e P de9ddd93604345dba2de5001ed497449: Deleting consensus metadata
I20260619 12:11:57.423480 15062 catalog_manager.cc:4841] TS de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): tablet 72e3da61c41b4eb183a8dab899693d6e (table test-table [id=6f1a663bd3e047ebaaa4a7f159e7c0e7]) successfully deleted
I20260619 12:11:57.423506 15063 catalog_manager.cc:4841] TS 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): tablet 72e3da61c41b4eb183a8dab899693d6e (table test-table [id=6f1a663bd3e047ebaaa4a7f159e7c0e7]) successfully deleted
W20260619 12:11:57.425293 15062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:36003 (error 108)
I20260619 12:11:57.425331 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15272
I20260619 12:11:57.426222 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.426085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:36003 (local address 127.0.0.1:50430)
0619 12:11:57.426142 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.426184 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 12:11:57.426362 15062 catalog_manager.cc:4584] TS 39b31ed224e74f2fac1396f7665b5b8f (127.0.201.1:36003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72e3da61c41b4eb183a8dab899693d6e: Network error: Client connection negotiation failed: client connection to 127.0.201.1:36003: connect: Connection refused (error 111)
I20260619 12:11:57.430696 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15407
I20260619 12:11:57.435118 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15045
I20260619 12:11:57.440640 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46683
--webserver_interface=127.0.201.62
--webserver_port=46689
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:57.520013 15567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:57.520226 15567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:57.520259 15567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:57.521368 15567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:11:57.521430 15567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:57.521453 15567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:57.521476 15567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:11:57.521505 15567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:11:57.522931 15567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:46683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=46689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:57.523181 15567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:57.523425 15567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:57.527989 15575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.527976 15573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.528187 15567 server_base.cc:1034] running on GCE node
W20260619 12:11:57.527968 15572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.528524 15567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:57.528797 15567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:57.529958 15567 hybrid_clock.cc:648] HybridClock initialized: now 1781871117529947 us; error 28 us; skew 500 ppm
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:57.532310 15567 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:11:57.532666 15567 webserver.cc:460] Webserver started at http://127.0.201.62:46689/ using document root <none> and password file <none>
I20260619 12:11:57.532850 15567 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:57.532891 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:57.534153 15567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:57.534896 15582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.535082 15567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:57.535185 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "b15f8925ffcc4226aa7b9adcf2f5803b"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "1724013940101a913fe94685e1822d1d"
server_key_iv: "7513b932d9bc24f4d247df2da4ff0e5c"
server_key_version: "encryptionkey@0"
I20260619 12:11:57.535591 15567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:57.554195 15567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:57.556524 15567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:57.556681 15567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:57.560489 15567 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:46683
I20260619 12:11:57.560540 15634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:46683 every 8 connection(s)
I20260619 12:11:57.560878 15567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:11:57.562320 15635 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:11:57.562893 15635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap starting.
I20260619 12:11:57.565186 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15567
I20260619 12:11:57.565490 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:36003
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:46683
--webserver_port=43265
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:57.571447 15635 log.cc:824] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:57.573570 15635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:11:57.573858 15635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b: Bootstrap complete.
I20260619 12:11:57.576097 15635 raft_consensus.cc:348] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:57.576327 15635 raft_consensus.cc:729] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15f8925ffcc4226aa7b9adcf2f5803b, State: Initialized, Role: FOLLOWER
I20260619 12:11:57.576529 15635 consensus_queue.cc:260] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:57.576625 15635 raft_consensus.cc:388] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:11:57.576678 15635 raft_consensus.cc:482] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:11:57.576723 15635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:11:57.577646 15635 raft_consensus.cc:504] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:57.577762 15635 leader_election.cc:302] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b15f8925ffcc4226aa7b9adcf2f5803b; no voters:
I20260619 12:11:57.577939 15635 leader_election.cc:288] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 12:11:57.577998 15639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:11:57.578148 15639 raft_consensus.cc:686] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [term 3 LEADER]: Becoming Leader. State: Replica: b15f8925ffcc4226aa7b9adcf2f5803b, State: Running, Role: LEADER
I20260619 12:11:57.578186 15635 sys_catalog.cc:564] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:11:57.578239 15639 consensus_queue.cc:237] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } }
I20260619 12:11:57.578503 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b15f8925ffcc4226aa7b9adcf2f5803b. Latest consensus state: current_term: 3 leader_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } } }
I20260619 12:11:57.578573 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:57.578506 15640 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15f8925ffcc4226aa7b9adcf2f5803b" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 46683 } } }
I20260619 12:11:57.578761 15640 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15f8925ffcc4226aa7b9adcf2f5803b [sys.catalog]: This master's current role is: LEADER
I20260619 12:11:57.578773 15645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:11:57.579746 15645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:11:57.579880 15645 catalog_manager.cc:1259] Loaded cluster ID: e8bed7320c7246d398aeb96abe673b1f
I20260619 12:11:57.579921 15645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 12:11:57.580226 15645 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f40a4ef49ad google::LogMessage::Fail() at ??:0
@ 0x7f40a4ef788a google::LogMessage::SendToLog() at ??:0
@ 0x7f40a4ef44de google::LogMessage::Flush() at ??:0
@ 0x7f40a4ef5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f40ace190f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f40ace228b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f40a6dc20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f40a6db7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f40a7c1b6db start_thread at ??:0
@ 0x7f40a39f261f clone at ??:0
W20260619 12:11:57.686290 15637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:57.686568 15637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:57.686605 15637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:57.688041 15637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:57.688090 15637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:57.688117 15637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:11:57.689548 15637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:36003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=43265
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:57.689775 15637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:57.690011 15637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:57.694813 15672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.694918 15671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.694839 15674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.694948 15637 server_base.cc:1034] running on GCE node
I20260619 12:11:57.695173 15637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:57.695449 15637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:57.696630 15637 hybrid_clock.cc:648] HybridClock initialized: now 1781871117696609 us; error 30 us; skew 500 ppm
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:57.699855 15637 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:11:57.700595 15637 webserver.cc:460] Webserver started at http://127.0.201.1:43265/ using document root <none> and password file <none>
I20260619 12:11:57.701159 15637 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:57.701275 15637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:57.704021 15637 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:11:57.704842 15681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.705835 15637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:57.705927 15637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "39b31ed224e74f2fac1396f7665b5b8f"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "555007d0bf3cabce02fdd45f5b91940d"
server_key_iv: "c911a07d85c92125ebb61e9c644495bf"
server_key_version: "encryptionkey@0"
I20260619 12:11:57.706373 15637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:57.730839 15637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:57.733494 15637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:57.733645 15637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:57.733907 15637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:57.734385 15688 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:11:57.735926 15637 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:11:57.735985 15637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 12:11:57.736017 15637 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:11:57.736865 15637 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:11:57.736910 15637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 12:11:57.737003 15688 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap starting.
I20260619 12:11:57.744585 15637 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:36003
I20260619 12:11:57.744743 15795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:36003 every 8 connection(s)
I20260619 12:11:57.745091 15637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:11:57.747682 15688 log.cc:824] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:11:57.749708 15688 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:11:57.749995 15688 tablet_bootstrap.cc:492] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Bootstrap complete.
I20260619 12:11:57.750912 15688 ts_tablet_manager.cc:1397] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260619 12:11:57.751477 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15637
I20260619 12:11:57.751770 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:44593
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:46683
--webserver_port=33637
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:11:57.752611 15688 raft_consensus.cc:348] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.752992 15688 raft_consensus.cc:729] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b31ed224e74f2fac1396f7665b5b8f, State: Initialized, Role: FOLLOWER
I20260619 12:11:57.753168 15688 consensus_queue.cc:260] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:57.753439 15688 ts_tablet_manager.cc:1428] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260619 12:11:57.811542 15799 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:11:57.747953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:59039)
0619 12:11:57.748141 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.748191 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:57.748800 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:57.811295 (+ 62495us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
W20260619 12:11:57.811825 15796 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:46683 (0 consecutive failures): Network error: Failed to ping master at 127.0.201.62:46683: Client connection negotiation failed: client connection to 127.0.201.62:46683: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:11:57.812199 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.812079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:50825)
0619 12:11:57.812130 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.812174 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:11:57.812525 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.812439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:41859)
0619 12:11:57.812482 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.812507 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 12:11:57.812636 15796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:11:57.832824 15800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:57.832993 15800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:57.833032 15800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:57.834084 15800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:57.834115 15800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:57.834141 15800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:11:57.835554 15800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:44593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=33637
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:57.835727 15800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:57.835886 15800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:11:57.840744 15800 server_base.cc:1034] running on GCE node
W20260619 12:11:57.840719 15809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.840700 15807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.840714 15806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.841133 15800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:57.841355 15800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:57.842520 15800 hybrid_clock.cc:648] HybridClock initialized: now 1781871117842509 us; error 25 us; skew 500 ppm
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:57.844944 15800 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:11:57.845355 15800 webserver.cc:460] Webserver started at http://127.0.201.2:33637/ using document root <none> and password file <none>
I20260619 12:11:57.845574 15800 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:57.845642 15800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:57.846854 15800 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:11:57.847468 15816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.847630 15800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.847735 15800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "8f90cda8f49a472388a42c2e9a36cced"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "98167e40c8cb7ff0462f29103c5d2a0e"
server_key_iv: "6d1380b69b314b63968ca0980de6cad0"
server_key_version: "encryptionkey@0"
I20260619 12:11:57.848039 15800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:57.864588 15800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:57.866329 15800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:57.866487 15800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:57.866689 15800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:57.866974 15800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:57.867028 15800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.867066 15800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:57.867095 15800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.872753 15800 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44593
I20260619 12:11:57.872802 15929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44593 every 8 connection(s)
I20260619 12:11:57.873129 15800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:11:57.874089 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.873738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:59621)
0619 12:11:57.873897 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.873958 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
W20260619 12:11:57.874481 15930 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:46683 (0 consecutive failures): Network error: Failed to ping master at 127.0.201.62:46683: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
I20260619 12:11:57.874748 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.874644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:59803)
0619 12:11:57.874691 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.874733 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 12:11:57.875034 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:57.874963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:42411)
0619 12:11:57.875000 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:57.875020 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 12:11:57.875144 15930 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:11:57.877445 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15800
I20260619 12:11:57.877693 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:44481
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:46683
--webserver_port=38519
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:11:57.957247 15933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:11:57.957427 15933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:11:57.957450 15933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:11:57.958575 15933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:11:57.958614 15933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:11:57.958639 15933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:11:57.960065 15933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:44481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=38519
--tserver_master_addrs=127.0.201.62:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:11:57.960294 15933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:11:57.960531 15933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:11:57.965622 15938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.965675 15939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:11:57.965643 15941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:11:57.965678 15933 server_base.cc:1034] running on GCE node
I20260619 12:11:57.966002 15933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:11:57.966217 15933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:11:57.967368 15933 hybrid_clock.cc:648] HybridClock initialized: now 1781871117967372 us; error 44 us; skew 500 ppm
Jun 19 12:11:57 dist-test-slave-c910 krb5kdc[14499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:11:57.969724 15933 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:11:57.970165 15933 webserver.cc:460] Webserver started at http://127.0.201.3:38519/ using document root <none> and password file <none>
I20260619 12:11:57.970387 15933 fs_manager.cc:359] Metadata directory not provided
I20260619 12:11:57.970436 15933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:11:57.971823 15933 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:57.972505 15948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:57.972709 15933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:11:57.972776 15933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "de9ddd93604345dba2de5001ed497449"
format_stamp: "Formatted at 2026-06-19 12:11:56 on dist-test-slave-c910"
server_key: "86267b2c3bb489646ca0b7285c017367"
server_key_iv: "3b74a92ca976bfd9fa6622b3939da621"
server_key_version: "encryptionkey@0"
I20260619 12:11:57.973114 15933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:11:58.000190 15933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:11:58.002027 15933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:11:58.002135 15933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:11:58.002358 15933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:11:58.002663 15933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:11:58.002715 15933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:58.002737 15933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:11:58.002750 15933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:11:58.008543 15933 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:44481
I20260619 12:11:58.008602 16061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:44481 every 8 connection(s)
I20260619 12:11:58.008957 15933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:11:58.010195 16064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.009722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:56575)
0619 12:11:58.009935 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.010038 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
W20260619 12:11:58.010695 16062 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:46683 (0 consecutive failures): Network error: Failed to ping master at 127.0.201.62:46683: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
I20260619 12:11:58.011009 16064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.010896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:33679)
0619 12:11:58.010941 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.010985 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:11:58.011313 16064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.011246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:55405)
0619 12:11:58.011273 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.011300 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 12:11:58.011494 16062 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:11:58.012457 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 15933
I20260619 12:11:58.813158 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.812948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:59645)
0619 12:11:58.813064 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.813097 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260619 12:11:58.875626 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.875434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:33595)
0619 12:11:58.875545 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.875578 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260619 12:11:58.923795 16068 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:11:58.923959 16068 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:11:58.924299 16068 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
Jun 19 12:11:58 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.2@KRBTEST.COM
Jun 19 12:11:58 dist-test-slave-c910 krb5kdc[14499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.3@KRBTEST.COM
I20260619 12:11:58.934672 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.924563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:50275 (local address 127.0.201.3:44481)
0619 12:11:58.924695 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:58.924708 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:58.924736 (+ 28us) server_negotiation.cc:407] Connection header received
0619 12:11:58.925178 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:58.925180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:58.925233 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:58.925358 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:58.926137 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.927010 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.928031 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.928433 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.928558 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:58.931284 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:58.931287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:58.931287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:58.931295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:58.933159 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933586 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:58.933588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:58.933589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:58.933660 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933863 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:58.933864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:58.933864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:58.934145 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:58.934221 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:58.934447 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 12:11:58.934571 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260619 12:11:58.934682 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.924662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:60487 (local address 127.0.201.2:44593)
0619 12:11:58.924795 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:11:58.924804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:11:58.924831 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:11:58.925241 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:11:58.925243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:11:58.925277 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:11:58.925425 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:11:58.926159 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.926999 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.928067 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.928433 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.928558 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:58.930833 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:11:58.930837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:11:58.930837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:11:58.930846 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:11:58.932828 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933337 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:58.933339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:58.933339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:58.933421 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933723 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:11:58.933724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:11:58.933724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:11:58.934022 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:11:58.934105 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:11:58.934433 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 12:11:58.934570 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
I20260619 12:11:58.934684 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.924530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44593 (local address 127.0.201.1:60487)
0619 12:11:58.924642 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.924652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:58.924687 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:58.925437 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:58.925438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:58.925836 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:58.926021 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:58.926027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.927097 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.927098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:58.927945 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:58.927947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.928523 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.928524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:58.928654 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:58.929265 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:58.929269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:58.930730 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:58.932927 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:58.932935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:58.932937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:58.933256 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:58.933520 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:58.933521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:58.933655 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:58.934096 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:58.934101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:58.934236 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 12:11:58.934355 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 12:11:58.934557 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":6,"thread_start_us":67,"threads_started":1}
I20260619 12:11:58.934710 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:58.924420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:44481 (local address 127.0.201.1:50275)
0619 12:11:58.924481 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:58.924494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:11:58.924563 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:11:58.925414 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:11:58.925417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:11:58.925823 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:11:58.926021 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:58.926026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.927125 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.927125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:58.927942 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:11:58.927944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:11:58.928524 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:11:58.928525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:11:58.928655 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:11:58.929210 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:11:58.929217 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:11:58.931183 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:11:58.933227 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:58.933229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:58.933507 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:58.933744 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:11:58.933745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:11:58.933745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:11:58.933801 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:11:58.934212 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:11:58.934213 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:11:58.934244 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:11:58.934388 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 12:11:58.934611 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":177}
W20260619 12:11:58.935858 15683 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
W20260619 12:11:58.935956 15682 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
I20260619 12:11:58.935999 15682 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f; no voters: 8f90cda8f49a472388a42c2e9a36cced, de9ddd93604345dba2de5001ed497449
I20260619 12:11:58.936107 16068 raft_consensus.cc:2726] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:11:59.012022 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:59.011793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:50835)
0619 12:11:59.011943 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:59.011985 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260619 12:11:59.813849 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:59.813609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:53985)
0619 12:11:59.813759 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:59.813799 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260619 12:11:59.876194 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:11:59.876009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:49745)
0619 12:11:59.876124 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:11:59.876158 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260619 12:12:00.012662 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:00.012458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:38775)
0619 12:12:00.012579 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:00.012620 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260619 12:12:00.802424 16076 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:00.802532 16076 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:12:00.802739 16076 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
W20260619 12:12:00.803215 15683 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
W20260619 12:12:00.803292 15682 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
I20260619 12:12:00.803337 15682 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f; no voters: 8f90cda8f49a472388a42c2e9a36cced, de9ddd93604345dba2de5001ed497449
I20260619 12:12:00.803429 16076 raft_consensus.cc:2726] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:12:00.814493 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:00.814272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:41483)
0619 12:12:00.814415 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:00.814454 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260619 12:12:00.876855 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:00.876655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:51971)
0619 12:12:00.876788 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:00.876818 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260619 12:12:01.013312 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:01.013125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:40703)
0619 12:12:01.013241 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:01.013273 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260619 12:12:01.815269 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:01.815003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:55373)
0619 12:12:01.815139 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:01.815171 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260619 12:12:01.877490 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:01.877278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:42415)
0619 12:12:01.877409 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:01.877447 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260619 12:12:02.014035 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:02.013847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:33587)
0619 12:12:02.013955 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:02.013993 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260619 12:12:02.815934 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:02.815735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:33761)
0619 12:12:02.815855 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:02.815890 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260619 12:12:02.878075 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:02.877899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:58591)
0619 12:12:02.878006 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:02.878037 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260619 12:12:03.014746 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:03.014521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:59867)
0619 12:12:03.014656 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:03.014694 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260619 12:12:03.240023 16086 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:03.240114 16086 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:12:03.240263 16086 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
W20260619 12:12:03.240646 15682 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
W20260619 12:12:03.240712 15683 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
I20260619 12:12:03.240901 15683 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f; no voters: 8f90cda8f49a472388a42c2e9a36cced, de9ddd93604345dba2de5001ed497449
I20260619 12:12:03.241019 16086 raft_consensus.cc:2726] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:12:03.816720 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:03.816421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:47459)
0619 12:12:03.816637 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:03.816674 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":57,"threads_started":1}
I20260619 12:12:03.878676 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:03.878492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:41133)
0619 12:12:03.878608 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:03.878640 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260619 12:12:04.015552 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:04.015370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:36535)
0619 12:12:04.015478 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:04.015511 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260619 12:12:04.817374 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:04.817161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:60183)
0619 12:12:04.817295 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:04.817327 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260619 12:12:04.879420 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:04.879120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:60617)
0619 12:12:04.879336 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:04.879368 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":177,"threads_started":1}
I20260619 12:12:05.016197 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:05.015988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:42145)
0619 12:12:05.016126 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:05.016157 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260619 12:12:05.818117 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:05.817853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:41761)
0619 12:12:05.818028 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:05.818069 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
I20260619 12:12:05.880205 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:05.879988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:36435)
0619 12:12:05.880132 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:05.880163 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260619 12:12:06.016916 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:06.016679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:49821)
0619 12:12:06.016831 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:06.016870 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260619 12:12:06.818817 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:06.818590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:51115)
0619 12:12:06.818740 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:06.818773 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260619 12:12:06.880815 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:06.880612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:50191)
0619 12:12:06.880746 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:06.880775 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 12:12:07.017550 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:07.017327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:51345)
0619 12:12:07.017476 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:07.017507 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260619 12:12:07.136660 16099 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:07.136760 16099 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:12:07.136979 16099 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
W20260619 12:12:07.137382 15683 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
W20260619 12:12:07.137458 15682 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
I20260619 12:12:07.137518 15682 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f; no voters: 8f90cda8f49a472388a42c2e9a36cced, de9ddd93604345dba2de5001ed497449
I20260619 12:12:07.137606 16099 raft_consensus.cc:2726] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:12:07.819487 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:07.819267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:36705)
0619 12:12:07.819410 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:07.819441 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260619 12:12:07.881431 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:07.881225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:35337)
0619 12:12:07.881359 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:07.881392 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260619 12:12:08.018234 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:08.018018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:52895)
0619 12:12:08.018153 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:08.018187 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 12:12:08.820237 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:08.820011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:53905)
0619 12:12:08.820153 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:08.820190 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260619 12:12:08.882073 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:08.881881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:54587)
0619 12:12:08.882000 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:08.882031 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260619 12:12:09.018882 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:09.018673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:49045)
0619 12:12:09.018809 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:09.018840 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260619 12:12:09.820904 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:09.820690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:47563)
0619 12:12:09.820820 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:09.820850 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260619 12:12:09.882687 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:09.882477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:58761)
0619 12:12:09.882595 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:09.882632 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260619 12:12:09.896883 16108 raft_consensus.cc:482] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:09.896970 16108 raft_consensus.cc:504] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9ddd93604345dba2de5001ed497449" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 44481 } } peers { permanent_uuid: "8f90cda8f49a472388a42c2e9a36cced" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44593 } } peers { permanent_uuid: "39b31ed224e74f2fac1396f7665b5b8f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 36003 } }
I20260619 12:12:09.897140 16108 leader_election.cc:288] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481), 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593)
W20260619 12:12:09.897526 15683 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f90cda8f49a472388a42c2e9a36cced (127.0.201.2:44593): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
W20260619 12:12:09.897610 15682 leader_election.cc:341] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de9ddd93604345dba2de5001ed497449 (127.0.201.3:44481): Not found: Tablet not found: 72e3da61c41b4eb183a8dab899693d6e
I20260619 12:12:09.897701 15682 leader_election.cc:302] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39b31ed224e74f2fac1396f7665b5b8f; no voters: 8f90cda8f49a472388a42c2e9a36cced, de9ddd93604345dba2de5001ed497449
I20260619 12:12:09.897778 16108 raft_consensus.cc:2726] T 72e3da61c41b4eb183a8dab899693d6e P 39b31ed224e74f2fac1396f7665b5b8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:12:10.019522 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:10.019296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:45863)
0619 12:12:10.019443 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:10.019478 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260619 12:12:10.821657 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:10.821433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:52491)
0619 12:12:10.821579 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:10.821614 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260619 12:12:10.883541 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:10.883325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:37955)
0619 12:12:10.883440 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:10.883474 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260619 12:12:11.020215 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:11.020009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:35377)
0619 12:12:11.020140 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:11.020171 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260619 12:12:11.822414 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:11.822153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:51131)
0619 12:12:11.822331 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:11.822365 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
I20260619 12:12:11.884199 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:11.883991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:59235)
0619 12:12:11.884122 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:11.884157 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260619 12:12:12.020888 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:12.020690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:36493)
0619 12:12:12.020810 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:12.020839 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260619 12:12:12.823158 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:12.822970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.1:46835)
0619 12:12:12.823085 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:12.823115 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260619 12:12:12.884899 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:12.884683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.2:47265)
0619 12:12:12.884824 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:12.884857 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260619 12:12:13.019079 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15637
I20260619 12:12:13.021618 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:13.021427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:46683 (local address 127.0.201.3:52187)
0619 12:12:13.021546 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:13.021579 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260619 12:12:13.023974 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15800
I20260619 12:12:13.028304 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 15933
2026-06-19T12:12:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18251 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:12:13 dist-test-slave-c910 krb5kdc[16121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:12:13 dist-test-slave-c910 krb5kdc[16121](info): set up 2 sockets
Jun 19 12:12:13 dist-test-slave-c910 krb5kdc[16121](info): commencing operation
krb5kdc: starting...
W20260619 12:12:14.059295 804 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:12:14.172188 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38065
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:14.251871 16134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:14.252054 16134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:14.252075 16134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:14.253288 16134 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:12:14.253327 16134 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:12:14.253347 16134 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:12:14.253358 16134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:12:14.253409 16134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:14.253427 16134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:14.253453 16134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:12:14.253481 16134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:12:14.254900 16134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:38065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:14.255138 16134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:14.255378 16134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:14.260389 16142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:14.260394 16139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:14.260402 16140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:14.260555 16134 server_base.cc:1034] running on GCE node
I20260619 12:12:14.260859 16134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:14.261127 16134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:14.262302 16134 hybrid_clock.cc:648] HybridClock initialized: now 1781871134262292 us; error 32 us; skew 500 ppm
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:14.264842 16134 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:12:14.265288 16134 webserver.cc:460] Webserver started at http://127.0.201.62:37287/ using document root <none> and password file <none>
I20260619 12:12:14.265520 16134 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:14.265587 16134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:14.265718 16134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:12:14.266746 16134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "fff9869b95164a598d3824103f1dc617"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "383f9957056bda2023292be536932693"
server_key_iv: "b26184564e4191250f7f711fc80a8185"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.267088 16134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "fff9869b95164a598d3824103f1dc617"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "383f9957056bda2023292be536932693"
server_key_iv: "b26184564e4191250f7f711fc80a8185"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.268443 16134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:14.269331 16149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.269512 16134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:14.269613 16134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "fff9869b95164a598d3824103f1dc617"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "383f9957056bda2023292be536932693"
server_key_iv: "b26184564e4191250f7f711fc80a8185"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.269713 16134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:14.288000 16134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:14.290244 16134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:14.290407 16134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:14.294457 16134 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:38065
I20260619 12:12:14.294493 16201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:38065 every 8 connection(s)
I20260619 12:12:14.294829 16134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:12:14.295727 16202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:14.296305 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16134
I20260619 12:12:14.296414 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:12:14.296594 804 external_mini_cluster.cc:1428] Setting key 1215b37d2f41f00a090301cf1cb90cb9
I20260619 12:12:14.298611 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap starting.
I20260619 12:12:14.299273 16202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:14.299556 16202 log.cc:824] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:14.300309 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: No bootstrap required, opened a new log
I20260619 12:12:14.302845 16202 raft_consensus.cc:348] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:14.302971 16202 raft_consensus.cc:374] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:14.302992 16202 raft_consensus.cc:729] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff9869b95164a598d3824103f1dc617, State: Initialized, Role: FOLLOWER
I20260619 12:12:14.303121 16202 consensus_queue.cc:260] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:14.303238 16202 raft_consensus.cc:388] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:12:14.303279 16202 raft_consensus.cc:482] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:12:14.303325 16202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:14.304013 16202 raft_consensus.cc:504] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:14.304131 16202 leader_election.cc:302] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fff9869b95164a598d3824103f1dc617; no voters:
I20260619 12:12:14.304286 16202 leader_election.cc:288] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:12:14.304409 16207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:12:14.304533 16202 sys_catalog.cc:564] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:12:14.304601 16207 raft_consensus.cc:686] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 LEADER]: Becoming Leader. State: Replica: fff9869b95164a598d3824103f1dc617, State: Running, Role: LEADER
I20260619 12:12:14.304742 16207 consensus_queue.cc:237] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:14.305088 16209 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fff9869b95164a598d3824103f1dc617. Latest consensus state: current_term: 1 leader_uuid: "fff9869b95164a598d3824103f1dc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } } }
I20260619 12:12:14.305270 16209 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: This master's current role is: LEADER
I20260619 12:12:14.305544 16216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:12:14.305099 16208 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fff9869b95164a598d3824103f1dc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } } }
I20260619 12:12:14.305986 16208 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: This master's current role is: LEADER
I20260619 12:12:14.306031 16216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:12:14.307638 16216 catalog_manager.cc:1347] Generated new cluster ID: 8f43804dc854413e87fe985abb006a08
I20260619 12:12:14.307683 16216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:12:14.307979 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.297092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.0.201.62:38065)
0619 12:12:14.297266 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.297273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.297297 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:12:14.297683 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.297688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.297733 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.297838 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:14.298638 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.299667 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.300668 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.301057 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.301143 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.303875 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:14.303878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:14.303878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:14.303887 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:14.306332 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.306681 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.306684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.306685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.306783 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.307016 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.307017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.307018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.307372 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:14.307535 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.307642 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.307778 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260619 12:12:14.324400 16216 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:12:14.324932 16216 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:12:14.330039 16216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Generated new TSK 0
I20260619 12:12:14.330260 16216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:12:14.365657 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38065
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:14.448738 16228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:14.448975 16228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:14.449014 16228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:14.450167 16228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:14.450235 16228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:14.450284 16228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:12:14.451946 16228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:14.452219 16228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:14.452442 16228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:14.457360 16234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:14.457404 16233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:14.457370 16228 server_base.cc:1034] running on GCE node
W20260619 12:12:14.457378 16236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:14.457769 16228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:14.458020 16228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:14.459159 16228 hybrid_clock.cc:648] HybridClock initialized: now 1781871134459136 us; error 40 us; skew 500 ppm
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:14.461651 16228 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:12:14.462013 16228 webserver.cc:460] Webserver started at http://127.0.201.1:39901/ using document root <none> and password file <none>
I20260619 12:12:14.462183 16228 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:14.462224 16228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:14.462306 16228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:12:14.463308 16228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "a284accc420738e7fa5381f1f514c148"
server_key_iv: "ca8533233b3ca9cb42234d9899a5df18"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.463626 16228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "a284accc420738e7fa5381f1f514c148"
server_key_iv: "ca8533233b3ca9cb42234d9899a5df18"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.464849 16228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:14.465540 16243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.465696 16228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.465791 16228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "a284accc420738e7fa5381f1f514c148"
server_key_iv: "ca8533233b3ca9cb42234d9899a5df18"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.465874 16228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:14.476606 16228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:14.478590 16228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:14.478740 16228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:14.478946 16228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:14.479305 16228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:14.479379 16228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.479420 16228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:14.479447 16228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.485160 16228 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:33261
I20260619 12:12:14.485201 16356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:33261 every 8 connection(s)
I20260619 12:12:14.485500 16228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:12:14.490600 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16228
I20260619 12:12:14.490688 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:12:14.490935 804 external_mini_cluster.cc:1428] Setting key 88ae86e6682d12cdd079abdbdf3eeb62
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:14.495594 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.486167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:54865 (local address 127.0.201.62:38065)
0619 12:12:14.486225 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.486227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.486432 (+ 205us) server_negotiation.cc:407] Connection header received
0619 12:12:14.486981 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.486982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.486996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.487068 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:14.488078 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.488612 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.489647 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.489870 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.489962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.492392 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:14.492393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:14.492394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:14.492403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:14.493977 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.494520 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.494521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.494521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.494574 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.494869 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.494870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.494870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.494969 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:14.495021 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.495395 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.495467 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:12:14.495841 16359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.486138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:54865)
0619 12:12:14.486319 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.486352 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.486874 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.487145 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.487149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.487778 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:14.487961 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.487966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.488738 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.488739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.489538 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.489540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.489972 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.489973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.490101 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.490617 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:14.490625 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:14.492296 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:14.494090 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:14.494098 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:14.494100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:14.494426 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:14.494646 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:14.494647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:14.494647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:14.494797 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:14.495038 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:14.495044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:14.495251 (+ 207us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.495381 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.495560 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260619 12:12:14.496417 16357 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38065
I20260619 12:12:14.496507 16357 heartbeater.cc:464] Registering TS with master...
I20260619 12:12:14.496676 16357 heartbeater.cc:511] Master 127.0.201.62:38065 requested a full tablet report, sending...
I20260619 12:12:14.497148 16166 ts_manager.cc:194] Registered new tserver with Master: fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261)
I20260619 12:12:14.497766 16166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:54865
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:12:14.517504 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38065
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:14.600163 16362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:14.600399 16362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:14.600437 16362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:14.601563 16362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:14.601630 16362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:14.601675 16362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:12:14.603096 16362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:14.603374 16362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:14.603614 16362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:14.608724 16368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:14.608765 16370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:14.608731 16367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:14.609032 16362 server_base.cc:1034] running on GCE node
I20260619 12:12:14.609259 16362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:14.609506 16362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:14.610673 16362 hybrid_clock.cc:648] HybridClock initialized: now 1781871134610655 us; error 29 us; skew 500 ppm
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:14.613144 16362 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:12:14.613595 16362 webserver.cc:460] Webserver started at http://127.0.201.2:39785/ using document root <none> and password file <none>
I20260619 12:12:14.613837 16362 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:14.613909 16362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:14.614050 16362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:12:14.615095 16362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "c660192d033542e5a7180a4463db3279"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "3bdd10e9e97d5a293a88d609a6b7e6c0"
server_key_iv: "d16dd4d468c4130c8f146986e426c968"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.615516 16362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "c660192d033542e5a7180a4463db3279"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "3bdd10e9e97d5a293a88d609a6b7e6c0"
server_key_iv: "d16dd4d468c4130c8f146986e426c968"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.616859 16362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:14.617579 16377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.617746 16362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.617868 16362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "c660192d033542e5a7180a4463db3279"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "3bdd10e9e97d5a293a88d609a6b7e6c0"
server_key_iv: "d16dd4d468c4130c8f146986e426c968"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.617942 16362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:14.635529 16362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:14.637269 16362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:14.637430 16362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:14.637656 16362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:14.637965 16362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:14.638019 16362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.638057 16362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:14.638085 16362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.643658 16362 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:36275
I20260619 12:12:14.643731 16490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:36275 every 8 connection(s)
I20260619 12:12:14.644068 16362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:14.651984 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16362
I20260619 12:12:14.652097 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:12:14.652310 804 external_mini_cluster.cc:1428] Setting key 11f73ac3c357700310a2fc238c9dccea
I20260619 12:12:14.653995 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.644879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54909 (local address 127.0.201.62:38065)
0619 12:12:14.644965 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.644967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.645149 (+ 182us) server_negotiation.cc:407] Connection header received
0619 12:12:14.645702 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.645712 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.645723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.645772 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:14.646699 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.647347 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.648357 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.648561 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.648646 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.650788 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:14.650789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:14.650789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:14.650797 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:14.652501 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.652977 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.652979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.652979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.653037 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.653282 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.653283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.653283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.653430 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:14.653503 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.653774 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.653855 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:12:14.654074 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.644838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:54909)
0619 12:12:14.645029 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.645063 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.645588 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.645855 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.645859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.646409 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:14.646595 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.646600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.647440 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.647442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.648249 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.648251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.648657 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.648658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.648777 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.649291 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:14.649299 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:14.650687 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:14.652602 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:14.652609 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:14.652611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:14.652906 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:14.653103 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:14.653104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:14.653104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:14.653208 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:14.653494 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:14.653498 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:14.653603 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.653736 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.653901 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260619 12:12:14.654595 16491 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38065
I20260619 12:12:14.654699 16491 heartbeater.cc:464] Registering TS with master...
I20260619 12:12:14.654873 16491 heartbeater.cc:511] Master 127.0.201.62:38065 requested a full tablet report, sending...
I20260619 12:12:14.655272 16166 ts_manager.cc:194] Registered new tserver with Master: c660192d033542e5a7180a4463db3279 (127.0.201.2:36275)
I20260619 12:12:14.655633 16166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:54909
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:12:14.677310 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38065
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:14.756443 16496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:14.756635 16496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:14.756659 16496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:14.757795 16496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:14.757839 16496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:14.757864 16496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:12:14.759298 16496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:14.759534 16496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:14.759758 16496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:14.764639 16504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:14.764672 16502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:14.764745 16496 server_base.cc:1034] running on GCE node
W20260619 12:12:14.764642 16501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:14.765105 16496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:14.765308 16496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:14.766450 16496 hybrid_clock.cc:648] HybridClock initialized: now 1781871134766438 us; error 29 us; skew 500 ppm
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:14.769064 16496 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:12:14.769560 16496 webserver.cc:460] Webserver started at http://127.0.201.3:35221/ using document root <none> and password file <none>
I20260619 12:12:14.769789 16496 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:14.769865 16496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:14.770031 16496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:12:14.771024 16496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "f211fca897f841e3798ebb1a057b0d6d"
server_key_iv: "f8f523ca10009225399a1a3db2c2c4e1"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.771401 16496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "f211fca897f841e3798ebb1a057b0d6d"
server_key_iv: "f8f523ca10009225399a1a3db2c2c4e1"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.772666 16496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:14.773382 16511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.773550 16496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:12:14.773644 16496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "f211fca897f841e3798ebb1a057b0d6d"
server_key_iv: "f8f523ca10009225399a1a3db2c2c4e1"
server_key_version: "encryptionkey@0"
I20260619 12:12:14.773728 16496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:14.788913 16496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:14.791064 16496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:14.791308 16496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:14.791580 16496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:14.791884 16496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:14.791937 16496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.791978 16496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:14.792006 16496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:14.797575 16496 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:39115
I20260619 12:12:14.797617 16624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:39115 every 8 connection(s)
I20260619 12:12:14.797916 16496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:12:14.801760 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16496
I20260619 12:12:14.801867 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:12:14.802069 804 external_mini_cluster.cc:1428] Setting key d83bd682bdd26bc953a491302f512747
Jun 19 12:12:14 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:14.807026 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.798559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:44879 (local address 127.0.201.62:38065)
0619 12:12:14.798619 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.798621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.798794 (+ 173us) server_negotiation.cc:407] Connection header received
0619 12:12:14.799326 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.799328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.799337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.799393 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:14.800263 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.800763 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.801684 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.802008 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.802085 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.804321 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:14.804322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:14.804323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:14.804329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:14.805637 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.806015 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.806016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.806017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.806081 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.806382 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.806383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.806383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.806505 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:14.806581 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.806859 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.806933 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:12:14.807237 16627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.798519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:44879)
0619 12:12:14.798669 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.798710 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.799159 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.799445 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.799449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.799986 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:14.800189 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.800193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.800839 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.800840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.801603 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.801605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.802088 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.802088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.802230 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.802775 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:14.802784 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:14.804243 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:14.805702 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:14.805707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:14.805709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:14.805929 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:14.806183 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:14.806184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:14.806184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:14.806312 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:14.806573 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:14.806578 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:14.806714 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.806836 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.806990 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260619 12:12:14.807737 16625 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38065
I20260619 12:12:14.807852 16625 heartbeater.cc:464] Registering TS with master...
I20260619 12:12:14.808055 16625 heartbeater.cc:511] Master 127.0.201.62:38065 requested a full tablet report, sending...
I20260619 12:12:14.808383 16166 ts_manager.cc:194] Registered new tserver with Master: 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115)
I20260619 12:12:14.808697 16166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:44879
I20260619 12:12:14.813658 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:12:14.820688 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.814914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.0.201.62:38065)
0619 12:12:14.814947 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.814949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.815069 (+ 120us) server_negotiation.cc:407] Connection header received
0619 12:12:14.815142 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.815143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.815154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.815240 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:14.815790 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.816249 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.817018 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.817200 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.817273 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.818373 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:14.818375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:14.818375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:14.818382 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:14.819504 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.819878 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.819879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.819879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.819938 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:14.820131 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:14.820132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:14.820133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:14.820279 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:14.820391 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.820507 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.820588 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 12:12:14.822067 16166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:12:14.823726 16166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:12:14.830065 16634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.826022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33261 (local address 127.0.0.1:59578)
0619 12:12:14.826354 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.826363 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.826416 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.826679 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.826680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.826681 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:14.826724 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.826725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.828050 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.828051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.828795 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.828796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.829734 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.829734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.829809 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.829835 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.829913 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.829973 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":1,"thread_start_us":53,"threads_started":1}
I20260619 12:12:14.830085 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.826160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.0.201.1:33261)
0619 12:12:14.826358 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.826364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.826448 (+ 84us) server_negotiation.cc:407] Connection header received
0619 12:12:14.826495 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.826497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.826513 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.826581 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:14.826812 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.827933 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.828896 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.829641 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.829718 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.829779 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.829939 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.829994 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260619 12:12:14.830253 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.825913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.0.201.3:39115)
0619 12:12:14.826466 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.826469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.826479 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:12:14.826516 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.826518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.826534 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.826603 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:14.826839 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.827601 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.828981 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.829649 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.829717 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.829778 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.830120 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.830178 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":91,"threads_started":1}
I20260619 12:12:14.830291 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.825823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36275 (local address 127.0.0.1:55164)
0619 12:12:14.826131 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.826175 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.826241 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.826586 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.826590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.826593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:14.826676 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.826678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.828039 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.828040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.829255 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.829257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.830053 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.830054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.830116 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.830120 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.830174 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.830220 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":33,"thread_start_us":61,"threads_started":1}
I20260619 12:12:14.830482 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.826016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55164 (local address 127.0.201.2:36275)
0619 12:12:14.826201 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.826206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.826221 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:12:14.826365 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.826368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.826383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.826474 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:14.826766 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.827927 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.829342 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.829907 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.829967 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.830021 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.830359 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.830401 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260619 12:12:14.830080 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.825823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39115 (local address 127.0.0.1:53372)
0619 12:12:14.826130 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.826175 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.826242 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.826711 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.826712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.826715 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:14.826771 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.826773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.827696 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.827697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.828906 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.828908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.829825 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.829833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.829911 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.829916 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.829969 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.830009 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":51,"threads_started":1}
I20260619 12:12:14.831241 16291 tablet_service.cc:1467] Processing CreateTablet for tablet 56b11024291d4e8a90da134f2e648d45 (DEFAULT_TABLE table=test-table [id=3e5d0b47801e4424a4d084fa78f9718d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:12:14.831270 16425 tablet_service.cc:1467] Processing CreateTablet for tablet 56b11024291d4e8a90da134f2e648d45 (DEFAULT_TABLE table=test-table [id=3e5d0b47801e4424a4d084fa78f9718d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:12:14.831437 16559 tablet_service.cc:1467] Processing CreateTablet for tablet 56b11024291d4e8a90da134f2e648d45 (DEFAULT_TABLE table=test-table [id=3e5d0b47801e4424a4d084fa78f9718d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:12:14.831573 16425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b11024291d4e8a90da134f2e648d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:14.831584 16291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b11024291d4e8a90da134f2e648d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:14.831693 16559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b11024291d4e8a90da134f2e648d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:14.833925 16639 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Bootstrap starting.
I20260619 12:12:14.834575 16639 tablet_bootstrap.cc:654] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:14.834815 16640 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Bootstrap starting.
I20260619 12:12:14.835247 16639 log.cc:824] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:14.835353 16641 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap starting.
I20260619 12:12:14.835552 16640 tablet_bootstrap.cc:654] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:14.835932 16641 tablet_bootstrap.cc:654] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:14.836064 16639 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: No bootstrap required, opened a new log
I20260619 12:12:14.836140 16639 ts_tablet_manager.cc:1397] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:12:14.836135 16640 log.cc:824] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:14.836529 16641 log.cc:824] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:14.836927 16640 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: No bootstrap required, opened a new log
I20260619 12:12:14.837042 16640 ts_tablet_manager.cc:1397] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:12:14.837214 16641 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: No bootstrap required, opened a new log
I20260619 12:12:14.837292 16641 ts_tablet_manager.cc:1397] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:12:14.838124 16639 raft_consensus.cc:348] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.838303 16639 raft_consensus.cc:374] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:14.838341 16639 raft_consensus.cc:729] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c660192d033542e5a7180a4463db3279, State: Initialized, Role: FOLLOWER
I20260619 12:12:14.838512 16639 consensus_queue.cc:260] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.838780 16639 ts_tablet_manager.cc:1428] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:12:14.838922 16491 heartbeater.cc:503] Master 127.0.201.62:38065 was elected leader, sending a full tablet report...
I20260619 12:12:14.839008 16640 raft_consensus.cc:348] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.839135 16640 raft_consensus.cc:374] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:14.839155 16640 raft_consensus.cc:729] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, State: Initialized, Role: FOLLOWER
I20260619 12:12:14.839366 16640 consensus_queue.cc:260] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.839454 16641 raft_consensus.cc:348] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.839684 16640 ts_tablet_manager.cc:1428] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:12:14.839708 16625 heartbeater.cc:503] Master 127.0.201.62:38065 was elected leader, sending a full tablet report...
I20260619 12:12:14.839761 16641 raft_consensus.cc:374] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:14.839994 16641 raft_consensus.cc:729] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc734ea500964d5bb3e0f8061c2a1c86, State: Initialized, Role: FOLLOWER
I20260619 12:12:14.840106 16641 consensus_queue.cc:260] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.840513 16641 ts_tablet_manager.cc:1428] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:12:14.840654 16357 heartbeater.cc:503] Master 127.0.201.62:38065 was elected leader, sending a full tablet report...
I20260619 12:12:14.883811 16645 raft_consensus.cc:482] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:14.883929 16645 raft_consensus.cc:504] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.884229 16645 leader_election.cc:288] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261)
I20260619 12:12:14.888197 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.884604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:37981 (local address 127.0.201.1:33261)
0619 12:12:14.884651 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.884653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.884782 (+ 129us) server_negotiation.cc:407] Connection header received
0619 12:12:14.884810 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.884811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.884821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.884900 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:14.885048 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.885723 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.887216 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.887804 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.887923 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.887934 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.888087 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.888126 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:12:14.888197 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.884517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:47307 (local address 127.0.201.3:39115)
0619 12:12:14.884569 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.884571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.884584 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:12:14.884615 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.884616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.884629 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.884686 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:14.884878 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.885666 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.887225 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.887805 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.887897 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.887909 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.888087 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.888126 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:12:14.888223 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.884410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39115 (local address 127.0.201.2:47307)
0619 12:12:14.884484 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.884497 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.884558 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.884696 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.884697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.884699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:14.884780 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.884781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.885744 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.885745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.887108 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.887111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.887921 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.887922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.888003 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.888005 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.888065 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.888128 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":60}
I20260619 12:12:14.888231 16648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.884515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33261 (local address 127.0.201.2:37981)
0619 12:12:14.884698 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.884709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:14.884767 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:14.884923 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:14.884924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:14.884925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:14.884977 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.884978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.885789 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.885789 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.887108 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:14.887111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.887930 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.887931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:14.888001 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.888005 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:12:14.888065 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:12:14.888128 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":98,"thread_start_us":131,"threads_started":1}
I20260619 12:12:14.888610 16311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b11024291d4e8a90da134f2e648d45" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" is_pre_election: true
I20260619 12:12:14.888669 16579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b11024291d4e8a90da134f2e648d45" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" is_pre_election: true
I20260619 12:12:14.888775 16311 raft_consensus.cc:2454] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 0.
I20260619 12:12:14.888809 16579 raft_consensus.cc:2454] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 0.
I20260619 12:12:14.889017 16378 leader_election.cc:302] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279; no voters:
I20260619 12:12:14.889183 16645 raft_consensus.cc:2781] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:12:14.889257 16645 raft_consensus.cc:482] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:12:14.889274 16645 raft_consensus.cc:3037] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:12:14.889999 16645 raft_consensus.cc:504] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.890131 16645 leader_election.cc:288] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 election: Requested vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261)
I20260619 12:12:14.890319 16579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b11024291d4e8a90da134f2e648d45" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
I20260619 12:12:14.890380 16311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b11024291d4e8a90da134f2e648d45" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
I20260619 12:12:14.890400 16579 raft_consensus.cc:3037] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:12:14.890476 16311 raft_consensus.cc:3037] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:12:14.891093 16579 raft_consensus.cc:2454] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 1.
I20260619 12:12:14.891273 16311 raft_consensus.cc:2454] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 1.
I20260619 12:12:14.891294 16378 leader_election.cc:302] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279; no voters:
I20260619 12:12:14.891400 16645 raft_consensus.cc:2781] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:12:14.891561 16645 raft_consensus.cc:686] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 1 LEADER]: Becoming Leader. State: Replica: c660192d033542e5a7180a4463db3279, State: Running, Role: LEADER
I20260619 12:12:14.891659 16645 consensus_queue.cc:237] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:14.892534 16166 catalog_manager.cc:5495] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 reported cstate change: term changed from 0 to 1, leader changed from <none> to c660192d033542e5a7180a4463db3279 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "c660192d033542e5a7180a4463db3279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } health_report { overall_health: UNKNOWN } } }
W20260619 12:12:14.894662 16492 tablet.cc:2369] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:12:14.905771 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.903483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.0.201.2:36275)
0619 12:12:14.903533 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:14.903535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:14.903544 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:12:14.903621 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:14.903622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:14.903644 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:14.903719 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:12:14.903861 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.904351 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.905068 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:14.905245 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:14.905330 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:14.905495 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:12:14.905591 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:12:14.905685 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:14.905706 (+ 21us) server_negotiation.cc:299] Negotiation successful
0619 12:12:14.905723 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:12:14.907361 16311 raft_consensus.cc:1261] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Refusing update from remote peer c660192d033542e5a7180a4463db3279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:12:14.907361 16579 raft_consensus.cc:1261] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Refusing update from remote peer c660192d033542e5a7180a4463db3279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:12:14.907686 16645 consensus_queue.cc:1035] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:12:14.907823 16649 consensus_queue.cc:1035] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:12:14.910070 16652 mvcc.cc:205] Tried to move back new op lower bound from 7298544168578011136 to 7298544168516972544. Current Snapshot: MvccSnapshot[applied={T|T < 7298544168578011136}]
I20260619 12:12:14.911367 16653 mvcc.cc:205] Tried to move back new op lower bound from 7298544168578011136 to 7298544168516972544. Current Snapshot: MvccSnapshot[applied={T|T < 7298544168578011136 or (T in {7298544168578011136})}]
I20260619 12:12:14.912427 16165 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:12:14.912525 16165 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:12:14.913311 16165 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Sending DeleteTablet for 3 replicas of tablet 56b11024291d4e8a90da134f2e648d45
I20260619 12:12:14.913655 16559 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b11024291d4e8a90da134f2e648d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:12:14 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:53372
I20260619 12:12:14.913662 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16228
I20260619 12:12:14.913655 16425 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b11024291d4e8a90da134f2e648d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:12:14 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:55164
I20260619 12:12:14.913863 16291 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b11024291d4e8a90da134f2e648d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:12:14 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:59578
I20260619 12:12:14.914304 16662 tablet_replica.cc:330] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: stopping tablet replica
I20260619 12:12:14.914412 16662 raft_consensus.cc:2229] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:12:14.914465 16661 tablet_replica.cc:330] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: stopping tablet replica
I20260619 12:12:14.914580 16661 raft_consensus.cc:2229] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:12:14.914584 16662 raft_consensus.cc:2258] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:12:14.914753 16661 raft_consensus.cc:2258] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:12:14.915112 16662 ts_tablet_manager.cc:1905] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:12:14.915150 16661 ts_tablet_manager.cc:1905] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:12:14.916811 16662 ts_tablet_manager.cc:1918] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:12:14.916821 16661 ts_tablet_manager.cc:1918] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:12:14.916889 16661 log.cc:1196] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/56b11024291d4e8a90da134f2e648d45
I20260619 12:12:14.916946 16662 log.cc:1196] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/56b11024291d4e8a90da134f2e648d45
I20260619 12:12:14.917236 16661 ts_tablet_manager.cc:1939] T 56b11024291d4e8a90da134f2e648d45 P c660192d033542e5a7180a4463db3279: Deleting consensus metadata
I20260619 12:12:14.917256 16662 ts_tablet_manager.cc:1939] T 56b11024291d4e8a90da134f2e648d45 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Deleting consensus metadata
I20260619 12:12:14.917629 16153 catalog_manager.cc:4841] TS c660192d033542e5a7180a4463db3279 (127.0.201.2:36275): tablet 56b11024291d4e8a90da134f2e648d45 (table test-table [id=3e5d0b47801e4424a4d084fa78f9718d]) successfully deleted
I20260619 12:12:14.917629 16150 catalog_manager.cc:4841] TS 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115): tablet 56b11024291d4e8a90da134f2e648d45 (table test-table [id=3e5d0b47801e4424a4d084fa78f9718d]) successfully deleted
W20260619 12:12:14.920452 16150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:33261 (error 108)
I20260619 12:12:14.920625 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16362
I20260619 12:12:14.921166 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:14.920774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33261 (local address 127.0.0.1:59592)
0619 12:12:14.921096 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:14.921127 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:33261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302}
W20260619 12:12:14.921654 16150 catalog_manager.cc:4584] TS fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56b11024291d4e8a90da134f2e648d45: Network error: Client connection negotiation failed: client connection to 127.0.201.1:33261: connect: Connection refused (error 111)
I20260619 12:12:14.925753 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16496
I20260619 12:12:14.930043 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16134
I20260619 12:12:34.935155 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38065
--webserver_interface=127.0.201.62
--webserver_port=37287
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:35.013988 16664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:35.014178 16664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:35.014200 16664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:35.015439 16664 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:12:35.015475 16664 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:12:35.015488 16664 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:12:35.015501 16664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:12:35.015512 16664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:35.015561 16664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:35.015578 16664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:12:35.015599 16664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:12:35.017007 16664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:38065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=37287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:35.017244 16664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:35.017472 16664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:12:35.026156 16664 server_base.cc:1034] running on GCE node
W20260619 12:12:35.026285 16671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:35.026183 16670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:35.026141 16673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:35.026712 16664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:35.027068 16664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:35.028210 16664 hybrid_clock.cc:648] HybridClock initialized: now 1781871155028193 us; error 31 us; skew 500 ppm
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:35.036751 16664 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:12:35.037230 16664 webserver.cc:460] Webserver started at http://127.0.201.62:37287/ using document root <none> and password file <none>
I20260619 12:12:35.037464 16664 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:35.037519 16664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:35.038946 16664 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:35.039706 16680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.039945 16664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:12:35.040021 16664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "fff9869b95164a598d3824103f1dc617"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "383f9957056bda2023292be536932693"
server_key_iv: "b26184564e4191250f7f711fc80a8185"
server_key_version: "encryptionkey@0"
I20260619 12:12:35.040431 16664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:35.054574 16664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:35.056936 16664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:35.057058 16664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:35.061326 16664 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:38065
I20260619 12:12:35.061363 16732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:38065 every 8 connection(s)
I20260619 12:12:35.061734 16664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:12:35.063166 16733 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:12:35.063844 16733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap starting.
I20260619 12:12:35.069923 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16664
I20260619 12:12:35.070248 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:33261
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:38065
--webserver_port=39901
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:12:35.071913 16733 log.cc:824] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:35.074420 16733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:12:35.074793 16733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap complete.
I20260619 12:12:35.078752 16733 raft_consensus.cc:348] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:35.079119 16733 raft_consensus.cc:729] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff9869b95164a598d3824103f1dc617, State: Initialized, Role: FOLLOWER
I20260619 12:12:35.079306 16733 consensus_queue.cc:260] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:35.079394 16733 raft_consensus.cc:388] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:12:35.079443 16733 raft_consensus.cc:482] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:12:35.079488 16733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:12:35.081007 16733 raft_consensus.cc:504] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:35.081125 16733 leader_election.cc:302] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fff9869b95164a598d3824103f1dc617; no voters:
I20260619 12:12:35.081291 16733 leader_election.cc:288] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:12:35.081359 16737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:12:35.081525 16733 sys_catalog.cc:564] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:12:35.081566 16737 raft_consensus.cc:686] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 LEADER]: Becoming Leader. State: Replica: fff9869b95164a598d3824103f1dc617, State: Running, Role: LEADER
I20260619 12:12:35.081749 16737 consensus_queue.cc:237] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:35.082051 16738 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fff9869b95164a598d3824103f1dc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } } }
I20260619 12:12:35.082127 16738 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: This master's current role is: LEADER
I20260619 12:12:35.082393 16739 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fff9869b95164a598d3824103f1dc617. Latest consensus state: current_term: 2 leader_uuid: "fff9869b95164a598d3824103f1dc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } } }
I20260619 12:12:35.082428 16747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:12:35.082453 16739 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: This master's current role is: LEADER
I20260619 12:12:35.083319 16747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:12:35.083469 16747 catalog_manager.cc:1259] Loaded cluster ID: 8f43804dc854413e87fe985abb006a08
I20260619 12:12:35.083714 16747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:12:35.084916 16747 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:12:35.085137 16747 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Loaded TSK: 0
I20260619 12:12:35.102952 16747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Generated new TSK 1
I20260619 12:12:35.103061 16747 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:12:35.156107 16735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:35.156343 16735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:35.156383 16735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:35.157613 16735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:35.157680 16735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:35.157742 16735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:12:35.159415 16735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:33261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=39901
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:35.159679 16735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:35.159920 16735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:35.165800 16761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:35.165849 16762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:35.165987 16735 server_base.cc:1034] running on GCE node
W20260619 12:12:35.166086 16764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:35.166348 16735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:35.166639 16735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:35.167816 16735 hybrid_clock.cc:648] HybridClock initialized: now 1781871155167790 us; error 40 us; skew 500 ppm
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:35.170683 16735 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:12:35.171137 16735 webserver.cc:460] Webserver started at http://127.0.201.1:39901/ using document root <none> and password file <none>
I20260619 12:12:35.171383 16735 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:35.171463 16735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:35.172752 16735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:35.173424 16771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.173614 16735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:35.173722 16735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "a284accc420738e7fa5381f1f514c148"
server_key_iv: "ca8533233b3ca9cb42234d9899a5df18"
server_key_version: "encryptionkey@0"
I20260619 12:12:35.174031 16735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:35.183969 16735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:35.186138 16735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:35.186292 16735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:35.186501 16735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:35.186889 16778 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:12:35.188808 16735 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:12:35.188866 16735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 12:12:35.188923 16735 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:12:35.189502 16735 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:12:35.189556 16735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 12:12:35.189635 16778 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap starting.
I20260619 12:12:35.196538 16735 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:33261
I20260619 12:12:35.196619 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:33261 every 8 connection(s)
I20260619 12:12:35.196938 16735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:12:35.200546 16778 log.cc:824] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:35.202802 16778 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:12:35.203090 16778 tablet_bootstrap.cc:492] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap complete.
I20260619 12:12:35.204208 16778 ts_tablet_manager.cc:1397] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260619 12:12:35.205536 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16735
I20260619 12:12:35.205446 16778 raft_consensus.cc:348] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.205782 16778 raft_consensus.cc:729] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc734ea500964d5bb3e0f8061c2a1c86, State: Initialized, Role: FOLLOWER
I20260619 12:12:35.205834 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:36275
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:38065
--webserver_port=39785
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:12:35.205909 16778 consensus_queue.cc:260] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.206158 16778 ts_tablet_manager.cc:1428] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:35.211769 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.197654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:47337)
0619 12:12:35.197831 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.197871 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.198284 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.201314 (+ 3030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.201318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.201693 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:35.201938 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.201944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.203645 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.203646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.204946 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.204948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.205827 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.205828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.205946 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.206517 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:35.206526 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:35.208094 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:35.210143 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:35.210149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:35.210151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:35.210375 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:35.210631 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:35.210632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:35.210632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:35.210740 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:35.211086 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:35.211092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:35.211300 (+ 208us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.211410 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.211574 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260619 12:12:35.211782 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.197722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47337 (local address 127.0.201.62:38065)
0619 12:12:35.200865 (+ 3143us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.200874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.200912 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:12:35.201043 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.201050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.201105 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.201257 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:35.202082 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.203516 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.205043 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.205642 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.205740 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.208233 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:35.208241 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:35.208241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:35.208251 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:35.210047 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.210471 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.210474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.210474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.210561 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.210807 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.210808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.210809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.211014 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:35.211105 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.211461 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.211574 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3088,"thread_start_us":80,"threads_started":1}
I20260619 12:12:35.212393 16886 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38065
I20260619 12:12:35.212476 16886 heartbeater.cc:464] Registering TS with master...
I20260619 12:12:35.212621 16886 heartbeater.cc:511] Master 127.0.201.62:38065 requested a full tablet report, sending...
I20260619 12:12:35.213289 16697 ts_manager.cc:194] Registered new tserver with Master: fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261)
I20260619 12:12:35.214401 16697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:47337
I20260619 12:12:35.215308 16886 heartbeater.cc:503] Master 127.0.201.62:38065 was elected leader, sending a full tablet report...
I20260619 12:12:35.218755 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.213817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33261 (local address 127.0.0.1:42478)
0619 12:12:35.215300 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.215336 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.215436 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.215691 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.215693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.215695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:35.215775 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.215777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.216553 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.216553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.217787 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.217789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.218426 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.218427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.218520 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.218533 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.218633 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.218684 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1455,"thread_start_us":83,"threads_started":1}
I20260619 12:12:35.218752 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.214049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.0.201.1:33261)
0619 12:12:35.214207 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.214212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.215396 (+ 1184us) server_negotiation.cc:407] Connection header received
0619 12:12:35.215538 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.215541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.215556 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.215607 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:35.215855 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.216479 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.217890 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.218330 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.218423 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.218476 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.218650 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.218692 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260619 12:12:35.219305 16820 tablet_service.cc:1514] Processing DeleteTablet for tablet 56b11024291d4e8a90da134f2e648d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:12:14 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:42478
I20260619 12:12:35.219798 16895 tablet_replica.cc:330] stopping tablet replica
I20260619 12:12:35.219895 16895 raft_consensus.cc:2229] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:12:35.219959 16895 raft_consensus.cc:2258] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:12:35.220237 16895 ts_tablet_manager.cc:1905] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:12:35.222330 16895 ts_tablet_manager.cc:1918] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:12:35.222411 16895 log.cc:1196] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal/wals/56b11024291d4e8a90da134f2e648d45
I20260619 12:12:35.222659 16895 ts_tablet_manager.cc:1939] T 56b11024291d4e8a90da134f2e648d45 P fc734ea500964d5bb3e0f8061c2a1c86: Deleting consensus metadata
I20260619 12:12:35.223098 16681 catalog_manager.cc:4841] TS fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261): tablet 56b11024291d4e8a90da134f2e648d45 (table test-table [id=3e5d0b47801e4424a4d084fa78f9718d]) successfully deleted
W20260619 12:12:35.296654 16892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:35.296846 16892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:35.296870 16892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:35.298094 16892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:35.298131 16892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:35.298157 16892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:12:35.299582 16892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:36275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=39785
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:35.299813 16892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:35.300038 16892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:35.304931 16900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:35.304939 16901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:35.304988 16892 server_base.cc:1034] running on GCE node
W20260619 12:12:35.304939 16903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:35.305353 16892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:35.305639 16892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:35.306813 16892 hybrid_clock.cc:648] HybridClock initialized: now 1781871155306803 us; error 29 us; skew 500 ppm
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:35.309319 16892 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:12:35.309733 16892 webserver.cc:460] Webserver started at http://127.0.201.2:39785/ using document root <none> and password file <none>
I20260619 12:12:35.309952 16892 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:35.310034 16892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:35.311470 16892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:35.312093 16910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.312258 16892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.312361 16892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "c660192d033542e5a7180a4463db3279"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "3bdd10e9e97d5a293a88d609a6b7e6c0"
server_key_iv: "d16dd4d468c4130c8f146986e426c968"
server_key_version: "encryptionkey@0"
I20260619 12:12:35.312698 16892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:35.337174 16892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:35.339017 16892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:35.339210 16892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:35.339447 16892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:35.339761 16892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:35.339828 16892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.339869 16892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:35.339897 16892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.345665 16892 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:36275
I20260619 12:12:35.345714 17023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:36275 every 8 connection(s)
I20260619 12:12:35.346031 16892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:12:35.351429 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 16892
I20260619 12:12:35.351774 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:39115
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:38065
--webserver_port=35221
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:35.356617 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.346686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:54249)
0619 12:12:35.346833 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.346869 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.347388 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.347736 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.347739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.348329 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:35.348513 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.348518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.349317 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.349318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.350118 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.350120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.350506 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.350507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.350621 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.351108 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:35.351116 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:35.353044 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:35.355007 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:35.355014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:35.355016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:35.355448 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:35.355677 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:35.355678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:35.355679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:35.355802 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:35.356110 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:35.356116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:35.356236 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.356317 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.356438 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260619 12:12:35.356631 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.346694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54249 (local address 127.0.201.62:38065)
0619 12:12:35.346765 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.346767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.346934 (+ 167us) server_negotiation.cc:407] Connection header received
0619 12:12:35.347571 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.347573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.347598 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.347663 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:35.348692 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.349227 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.350212 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.350406 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.350468 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.353265 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:35.353267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:35.353267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:35.353276 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:35.354919 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.355519 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.355521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.355521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.355586 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.355881 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.355882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.355882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.356041 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:35.356115 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.356379 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.356482 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:12:35.357201 17024 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38065
I20260619 12:12:35.357316 17024 heartbeater.cc:464] Registering TS with master...
I20260619 12:12:35.357517 17024 heartbeater.cc:511] Master 127.0.201.62:38065 requested a full tablet report, sending...
I20260619 12:12:35.357928 16697 ts_manager.cc:194] Registered new tserver with Master: c660192d033542e5a7180a4463db3279 (127.0.201.2:36275)
I20260619 12:12:35.358331 16697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:54249
W20260619 12:12:35.433708 17027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:35.433899 17027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:35.433923 17027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:35.435075 17027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:35.435120 17027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:35.435148 17027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:12:35.436630 17027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:39115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=35221
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:35.436865 17027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:35.437119 17027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:35.442116 17032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:35.442263 17033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:35.442128 17035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:35.442363 17027 server_base.cc:1034] running on GCE node
I20260619 12:12:35.442585 17027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:35.442847 17027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:35.444000 17027 hybrid_clock.cc:648] HybridClock initialized: now 1781871155443988 us; error 31 us; skew 500 ppm
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:35.446376 17027 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:12:35.446818 17027 webserver.cc:460] Webserver started at http://127.0.201.3:35221/ using document root <none> and password file <none>
I20260619 12:12:35.447039 17027 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:35.447122 17027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:35.448482 17027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:35.449113 17042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.449295 17027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:35.449378 17027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "f211fca897f841e3798ebb1a057b0d6d"
server_key_iv: "f8f523ca10009225399a1a3db2c2c4e1"
server_key_version: "encryptionkey@0"
I20260619 12:12:35.449698 17027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:35.478724 17027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:35.480705 17027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:35.480854 17027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:35.481070 17027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:35.481369 17027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:35.481420 17027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.481491 17027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:35.481524 17027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.487238 17027 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:39115
I20260619 12:12:35.487273 17155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:39115 every 8 connection(s)
I20260619 12:12:35.487586 17027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:12:35 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:12:35.496138 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.488263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:53165 (local address 127.0.201.62:38065)
0619 12:12:35.488334 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.488336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.488547 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:12:35.489043 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.489044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.489057 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.489119 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:35.490008 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.490496 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.491475 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.491698 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.491782 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.493621 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:35.493622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:35.493622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:35.493629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:35.494759 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.495164 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.495165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.495165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.495259 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.495492 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.495493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.495494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.495607 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:35.495691 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.495931 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.496019 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:12:35.496316 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.488266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:53165)
0619 12:12:35.488431 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.488464 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.488941 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.489193 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.489196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.489721 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:35.489899 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.489904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.490585 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.490586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.491385 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.491387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.491792 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.491793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.491907 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.492286 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:35.492293 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:35.493526 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:35.494869 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:35.494876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:35.494878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:35.495082 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:35.495320 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:35.495321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:35.495322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:35.495425 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:35.495673 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:35.495679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:35.495808 (+ 129us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.495909 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.496083 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260619 12:12:35.496778 17156 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38065
I20260619 12:12:35.496846 17156 heartbeater.cc:464] Registering TS with master...
I20260619 12:12:35.497030 17156 heartbeater.cc:511] Master 127.0.201.62:38065 requested a full tablet report, sending...
I20260619 12:12:35.497440 16697 ts_manager.cc:194] Registered new tserver with Master: 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115)
I20260619 12:12:35.497671 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17027
I20260619 12:12:35.497975 16697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:53165
I20260619 12:12:35.503831 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.498030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.0.201.62:38065)
0619 12:12:35.498068 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.498070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.498271 (+ 201us) server_negotiation.cc:407] Connection header received
0619 12:12:35.498314 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.498314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.498324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.498385 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:35.498946 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.499449 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.500198 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.500363 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.500407 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.501714 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:35.501715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:35.501715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:35.501721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:35.502856 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.503252 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.503253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.503253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.503310 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.503457 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.503458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.503458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.503543 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:35.503650 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.503728 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.503779 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:12:35.504552 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:12:35.506815 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.504888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.0.201.62:38065)
0619 12:12:35.504925 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.504927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.505012 (+ 85us) server_negotiation.cc:407] Connection header received
0619 12:12:35.505044 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.505044 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.505071 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.505132 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:12:35.505219 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.505638 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.506311 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.506481 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.506536 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.506639 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:12:35.506673 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 12:12:35.506778 (+ 105us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48114: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 12:12:35.506914 16889 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48114: token expired
W20260619 12:12:35.506999 16628 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.201.62:38065)
I20260619 12:12:35.512239 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.507259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.0.201.62:38065)
0619 12:12:35.507290 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.507292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.507307 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:12:35.507352 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.507353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.507360 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.507415 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:35.507822 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.508267 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.508901 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.509096 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.509154 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.510097 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:35.510098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:35.510098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:35.510104 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:35.511371 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.511658 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.511659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.511659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.511710 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:35.511837 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:35.511837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:35.511837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:35.511944 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:35.512046 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.512131 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.512188 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 12:12:35.513247 16697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:12:35.514926 16697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:12:35.517177 16820 tablet_service.cc:1467] Processing CreateTablet for tablet 4215b91e72c1434498430912d7655bb8 (DEFAULT_TABLE table=test-table [id=aacdb44e289c424caf2b23b2261894aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:12:35.517356 16820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4215b91e72c1434498430912d7655bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:35.518877 16778 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap starting.
I20260619 12:12:35.519152 16778 tablet_bootstrap.cc:654] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:35.519726 16778 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: No bootstrap required, opened a new log
I20260619 12:12:35.519789 16778 ts_tablet_manager.cc:1397] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:35.519896 16778 raft_consensus.cc:348] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.519941 16778 raft_consensus.cc:374] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:35.519965 16778 raft_consensus.cc:729] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc734ea500964d5bb3e0f8061c2a1c86, State: Initialized, Role: FOLLOWER
I20260619 12:12:35.520021 16778 consensus_queue.cc:260] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.520116 16778 ts_tablet_manager.cc:1428] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:35.521049 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.516836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36275 (local address 127.0.0.1:44742)
0619 12:12:35.516879 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.516888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.516924 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.517334 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.517335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.517336 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:35.517372 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.517373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.518739 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.518740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.519937 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.519941 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.520839 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.520840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.520886 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.520889 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.520939 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.521001 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":39}
I20260619 12:12:35.521113 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.516960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.0.201.2:36275)
0619 12:12:35.517151 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.517154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.517164 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:12:35.517197 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.517199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.517211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.517321 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:35.517523 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.518635 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.520058 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.520768 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.520826 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.520862 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.520967 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.521002 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260619 12:12:35.521770 17161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.516835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39115 (local address 127.0.0.1:59822)
0619 12:12:35.516996 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.517005 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.517039 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.517634 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.517635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.517638 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:35.517750 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.517753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.518722 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.518723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.519959 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.519961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.521586 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.521587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.521637 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.521640 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.521692 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.521721 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":57,"thread_start_us":48,"threads_started":1}
I20260619 12:12:35.521804 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.516946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.0.201.3:39115)
0619 12:12:35.517476 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.517480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.517490 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:12:35.517532 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.517534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.517545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.517635 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:35.517818 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.518623 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.520889 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.521503 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.521575 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.521625 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.521702 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.521735 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":101,"threads_started":1}
I20260619 12:12:35.521917 16958 tablet_service.cc:1467] Processing CreateTablet for tablet 4215b91e72c1434498430912d7655bb8 (DEFAULT_TABLE table=test-table [id=aacdb44e289c424caf2b23b2261894aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:12:35.522233 16958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4215b91e72c1434498430912d7655bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:35.522493 17090 tablet_service.cc:1467] Processing CreateTablet for tablet 4215b91e72c1434498430912d7655bb8 (DEFAULT_TABLE table=test-table [id=aacdb44e289c424caf2b23b2261894aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:12:35.522801 17090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4215b91e72c1434498430912d7655bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:12:35.524860 17165 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Bootstrap starting.
I20260619 12:12:35.525583 17165 tablet_bootstrap.cc:654] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:35.525653 17166 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Bootstrap starting.
I20260619 12:12:35.526275 17165 log.cc:824] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:35.526361 17166 tablet_bootstrap.cc:654] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Neither blocks nor log segments found. Creating new log.
I20260619 12:12:35.526942 17166 log.cc:824] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:35.527119 17165 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: No bootstrap required, opened a new log
I20260619 12:12:35.527221 17165 ts_tablet_manager.cc:1397] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:12:35.527768 17166 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: No bootstrap required, opened a new log
I20260619 12:12:35.527842 17166 ts_tablet_manager.cc:1397] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:12:35.529338 17165 raft_consensus.cc:348] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.529532 17165 raft_consensus.cc:374] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:35.529588 17165 raft_consensus.cc:729] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c660192d033542e5a7180a4463db3279, State: Initialized, Role: FOLLOWER
I20260619 12:12:35.529709 17165 consensus_queue.cc:260] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.529842 17166 raft_consensus.cc:348] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.530000 17024 heartbeater.cc:503] Master 127.0.201.62:38065 was elected leader, sending a full tablet report...
I20260619 12:12:35.530018 17166 raft_consensus.cc:374] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:12:35.530061 17166 raft_consensus.cc:729] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, State: Initialized, Role: FOLLOWER
I20260619 12:12:35.530005 17165 ts_tablet_manager.cc:1428] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:12:35.530227 17166 consensus_queue.cc:260] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.530527 17166 ts_tablet_manager.cc:1428] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:12:35.530638 17156 heartbeater.cc:503] Master 127.0.201.62:38065 was elected leader, sending a full tablet report...
W20260619 12:12:35.596586 17025 tablet.cc:2369] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:12:35.671278 17169 raft_consensus.cc:482] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:35.671396 17169 raft_consensus.cc:504] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.671808 17169 leader_election.cc:288] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261)
I20260619 12:12:35.675099 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.672107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33261 (local address 127.0.201.2:45139)
0619 12:12:35.672265 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.672274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.672341 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.672494 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.672494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.672496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:35.672557 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.672559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.673457 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.673458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.674258 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.674260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.674845 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.674846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.674915 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.674919 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.674969 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.674999 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":19,"thread_start_us":88,"threads_started":1}
I20260619 12:12:35.675123 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.672208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:45139 (local address 127.0.201.1:33261)
0619 12:12:35.672250 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.672252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.672345 (+ 93us) server_negotiation.cc:407] Connection header received
0619 12:12:35.672417 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.672418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.672428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.672483 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:35.672627 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.673368 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.674350 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.674758 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.674817 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.674849 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.674996 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.675047 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:12:35.675175 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.672089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:57555 (local address 127.0.201.3:39115)
0619 12:12:35.672141 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.672143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.672154 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:12:35.672268 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.672270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.672283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.672374 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:12:35.672579 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.673346 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.674418 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.674771 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.674856 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.674865 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.675054 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.675092 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:12:35.675251 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.671989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39115 (local address 127.0.201.2:57555)
0619 12:12:35.672063 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.672080 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:35.672147 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:35.672395 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:35.672397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:35.672398 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:12:35.672502 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.672504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.673439 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.673441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.674317 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:35.674320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.674870 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.674871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:35.674947 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.674976 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 12:12:35.675048 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:12:35.675086 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":29}
I20260619 12:12:35.675724 17110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4215b91e72c1434498430912d7655bb8" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" is_pre_election: true
I20260619 12:12:35.675720 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4215b91e72c1434498430912d7655bb8" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" is_pre_election: true
I20260619 12:12:35.675873 17110 raft_consensus.cc:2454] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 0.
I20260619 12:12:35.675884 16840 raft_consensus.cc:2454] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 0.
I20260619 12:12:35.676100 16911 leader_election.cc:302] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c660192d033542e5a7180a4463db3279, fc734ea500964d5bb3e0f8061c2a1c86; no voters:
I20260619 12:12:35.676252 17169 raft_consensus.cc:2781] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:12:35.676307 17169 raft_consensus.cc:482] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:12:35.676342 17169 raft_consensus.cc:3037] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:12:35.676980 17169 raft_consensus.cc:504] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.677098 17169 leader_election.cc:288] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 election: Requested vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261)
I20260619 12:12:35.677255 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4215b91e72c1434498430912d7655bb8" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
I20260619 12:12:35.677243 17110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4215b91e72c1434498430912d7655bb8" candidate_uuid: "c660192d033542e5a7180a4463db3279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
I20260619 12:12:35.677333 17110 raft_consensus.cc:3037] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:12:35.677333 16840 raft_consensus.cc:3037] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:12:35.678251 17110 raft_consensus.cc:2454] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 1.
I20260619 12:12:35.678439 16911 leader_election.cc:302] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279; no voters:
I20260619 12:12:35.678562 17169 raft_consensus.cc:2781] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:12:35.678629 16840 raft_consensus.cc:2454] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c660192d033542e5a7180a4463db3279 in term 1.
I20260619 12:12:35.678748 17169 raft_consensus.cc:686] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 1 LEADER]: Becoming Leader. State: Replica: c660192d033542e5a7180a4463db3279, State: Running, Role: LEADER
I20260619 12:12:35.678849 17169 consensus_queue.cc:237] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:35.679739 16697 catalog_manager.cc:5495] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 reported cstate change: term changed from 0 to 1, leader changed from <none> to c660192d033542e5a7180a4463db3279 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "c660192d033542e5a7180a4463db3279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } health_report { overall_health: UNKNOWN } } }
W20260619 12:12:35.699709 16887 tablet.cc:2369] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:12:35.701465 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.698306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.0.201.2:36275)
0619 12:12:35.698385 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:35.698386 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:35.698442 (+ 56us) server_negotiation.cc:407] Connection header received
0619 12:12:35.698482 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:35.698483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:35.698499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:35.698577 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:12:35.698878 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.699770 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.700556 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:35.700889 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:35.700965 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:35.701131 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:12:35.701236 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:12:35.701328 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:35.701347 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 12:12:35.701366 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:12:35.703451 16840 raft_consensus.cc:1261] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Refusing update from remote peer c660192d033542e5a7180a4463db3279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:12:35.703474 17110 raft_consensus.cc:1261] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Refusing update from remote peer c660192d033542e5a7180a4463db3279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:12:35.703778 17172 consensus_queue.cc:1035] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:12:35.703914 17169 consensus_queue.cc:1035] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:12:35.706597 17174 mvcc.cc:205] Tried to move back new op lower bound from 7298544253758468096 to 7298544253661347840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544253758468096}]
I20260619 12:12:35.707008 17175 mvcc.cc:205] Tried to move back new op lower bound from 7298544253758468096 to 7298544253661347840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544253758468096}]
I20260619 12:12:35.707798 17177 mvcc.cc:205] Tried to move back new op lower bound from 7298544253758468096 to 7298544253661347840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544253758468096}]
I20260619 12:12:35.709017 16696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:12:35.709107 16696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:12:35.709849 16696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Sending DeleteTablet for 3 replicas of tablet 4215b91e72c1434498430912d7655bb8
I20260619 12:12:35.710201 17090 tablet_service.cc:1514] Processing DeleteTablet for tablet 4215b91e72c1434498430912d7655bb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:12:35 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:59822
I20260619 12:12:35.710184 16958 tablet_service.cc:1514] Processing DeleteTablet for tablet 4215b91e72c1434498430912d7655bb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:12:35 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:44742
I20260619 12:12:35.710268 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16735
I20260619 12:12:35.710971 17184 tablet_replica.cc:330] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: stopping tablet replica
I20260619 12:12:35.711086 17184 raft_consensus.cc:2229] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:12:35.711184 17184 raft_consensus.cc:2258] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:12:35.711308 17185 tablet_replica.cc:330] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: stopping tablet replica
I20260619 12:12:35.711433 17185 raft_consensus.cc:2229] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:12:35.711522 17185 raft_consensus.cc:2258] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:12:35.711856 17185 ts_tablet_manager.cc:1905] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:12:35.713514 17185 ts_tablet_manager.cc:1918] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:12:35.713583 17185 log.cc:1196] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/4215b91e72c1434498430912d7655bb8
I20260619 12:12:35.713845 17185 ts_tablet_manager.cc:1939] T 4215b91e72c1434498430912d7655bb8 P c660192d033542e5a7180a4463db3279: Deleting consensus metadata
I20260619 12:12:35.714231 16684 catalog_manager.cc:4841] TS c660192d033542e5a7180a4463db3279 (127.0.201.2:36275): tablet 4215b91e72c1434498430912d7655bb8 (table test-table [id=aacdb44e289c424caf2b23b2261894aa]) successfully deleted
I20260619 12:12:35.715595 17184 ts_tablet_manager.cc:1905] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:12:35.716282 16681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:33261 (error 108)
I20260619 12:12:35.716455 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16892
I20260619 12:12:35.717904 17184 ts_tablet_manager.cc:1918] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:12:35.717988 17184 log.cc:1196] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/4215b91e72c1434498430912d7655bb8
I20260619 12:12:35.718333 17184 ts_tablet_manager.cc:1939] T 4215b91e72c1434498430912d7655bb8 P 5bc6ac03f8d1449ebf5aeb2a3654bfa7: Deleting consensus metadata
I20260619 12:12:35.718477 17161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:35.718340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:33261 (local address 127.0.0.1:42494)
0619 12:12:35.718405 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:35.718446 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:33261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 12:12:35.718629 16681 catalog_manager.cc:4584] TS fc734ea500964d5bb3e0f8061c2a1c86 (127.0.201.1:33261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4215b91e72c1434498430912d7655bb8: Network error: Client connection negotiation failed: client connection to 127.0.201.1:33261: connect: Connection refused (error 111)
I20260619 12:12:35.719063 16681 catalog_manager.cc:4841] TS 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115): tablet 4215b91e72c1434498430912d7655bb8 (table test-table [id=aacdb44e289c424caf2b23b2261894aa]) successfully deleted
I20260619 12:12:35.721112 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17027
I20260619 12:12:35.726099 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 16664
I20260619 12:12:55.731513 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38065
--webserver_interface=127.0.201.62
--webserver_port=37287
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:55.811352 17187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:55.811548 17187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:55.811570 17187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:55.812745 17187 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:12:55.812785 17187 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:12:55.812801 17187 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:12:55.812814 17187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:12:55.812865 17187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:55.812893 17187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:55.812918 17187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:12:55.812942 17187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:12:55.814513 17187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:38065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=37287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:55.814761 17187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:55.815003 17187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:12:55.820003 17187 server_base.cc:1034] running on GCE node
W20260619 12:12:55.820037 17192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:55.819976 17195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:55.819976 17193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:55.820461 17187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:55.820694 17187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:55.821837 17187 hybrid_clock.cc:648] HybridClock initialized: now 1781871175821818 us; error 34 us; skew 500 ppm
Jun 19 12:12:55 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:55.828029 17187 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:12:55.828496 17187 webserver.cc:460] Webserver started at http://127.0.201.62:37287/ using document root <none> and password file <none>
I20260619 12:12:55.828692 17187 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:55.828733 17187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:55.830132 17187 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:55.830818 17202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:55.831030 17187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:12:55.831126 17187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "fff9869b95164a598d3824103f1dc617"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "383f9957056bda2023292be536932693"
server_key_iv: "b26184564e4191250f7f711fc80a8185"
server_key_version: "encryptionkey@0"
I20260619 12:12:55.831537 17187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:55.855173 17187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:55.857757 17187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:55.857913 17187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:55.863096 17187 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:38065
I20260619 12:12:55.863133 17254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:38065 every 8 connection(s)
I20260619 12:12:55.863607 17187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:12:55.865811 17255 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:12:55.866780 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17187
I20260619 12:12:55.866861 17255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap starting.
I20260619 12:12:55.867134 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:33261
--local_ip_for_outbound_sockets=127.0.201.1
--tserver_master_addrs=127.0.201.62:38065
--webserver_port=39901
--webserver_interface=127.0.201.1
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:12:55.876322 17255 log.cc:824] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:55.878471 17255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:12:55.878785 17255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Bootstrap complete.
I20260619 12:12:55.881085 17255 raft_consensus.cc:348] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:55.881379 17255 raft_consensus.cc:729] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff9869b95164a598d3824103f1dc617, State: Initialized, Role: FOLLOWER
I20260619 12:12:55.881587 17255 consensus_queue.cc:260] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:55.881667 17255 raft_consensus.cc:388] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:12:55.881719 17255 raft_consensus.cc:482] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:12:55.881767 17255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:12:55.882710 17255 raft_consensus.cc:504] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:55.882828 17255 leader_election.cc:302] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fff9869b95164a598d3824103f1dc617; no voters:
I20260619 12:12:55.883034 17255 leader_election.cc:288] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 12:12:55.883075 17259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:12:55.883206 17259 raft_consensus.cc:686] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [term 3 LEADER]: Becoming Leader. State: Replica: fff9869b95164a598d3824103f1dc617, State: Running, Role: LEADER
I20260619 12:12:55.883316 17255 sys_catalog.cc:564] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:12:55.883356 17259 consensus_queue.cc:237] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } }
I20260619 12:12:55.883628 17259 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fff9869b95164a598d3824103f1dc617. Latest consensus state: current_term: 3 leader_uuid: "fff9869b95164a598d3824103f1dc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } } }
I20260619 12:12:55.883632 17260 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fff9869b95164a598d3824103f1dc617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff9869b95164a598d3824103f1dc617" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38065 } } }
I20260619 12:12:55.883709 17259 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: This master's current role is: LEADER
I20260619 12:12:55.883770 17260 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617 [sys.catalog]: This master's current role is: LEADER
I20260619 12:12:55.883992 17263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:12:55.884835 17263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:12:55.885078 17263 catalog_manager.cc:1259] Loaded cluster ID: 8f43804dc854413e87fe985abb006a08
I20260619 12:12:55.885219 17263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:12:55.886399 17263 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:12:55.886615 17263 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Loaded TSK: 0
I20260619 12:12:55.886689 17263 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fff9869b95164a598d3824103f1dc617: Loaded TSK: 1
F20260619 12:12:55.886935 17263 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7efefb14f9ad google::LogMessage::Fail() at ??:0
@ 0x7efefb15288a google::LogMessage::SendToLog() at ??:0
@ 0x7efefb14f4de google::LogMessage::Flush() at ??:0
@ 0x7efefb150a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7efefd8bc104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7efefd8b9d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7eff030ad1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7eff030ad48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7eff030ad594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7eff03073c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7eff0307dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7efefd01d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7efefd012954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7efefde766db start_thread at ??:0
@ 0x7efef9c4d61f clone at ??:0
W20260619 12:12:55.969882 17257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:55.970060 17257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:55.970082 17257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:55.971287 17257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:55.971326 17257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:55.971354 17257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:12:55.972806 17257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:33261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=39901
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:55.973037 17257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:55.973266 17257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:55.978009 17297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:55.978044 17296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:55.978257 17299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:55.978377 17257 server_base.cc:1034] running on GCE node
I20260619 12:12:55.978574 17257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:55.978829 17257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:55.980010 17257 hybrid_clock.cc:648] HybridClock initialized: now 1781871175979986 us; error 37 us; skew 500 ppm
Jun 19 12:12:55 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:55.982867 17257 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:12:55.983423 17257 webserver.cc:460] Webserver started at http://127.0.201.1:39901/ using document root <none> and password file <none>
I20260619 12:12:55.983709 17257 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:55.983775 17257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:55.985571 17257 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:12:55.986377 17306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:55.986603 17257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:55.986687 17257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "fc734ea500964d5bb3e0f8061c2a1c86"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "a284accc420738e7fa5381f1f514c148"
server_key_iv: "ca8533233b3ca9cb42234d9899a5df18"
server_key_version: "encryptionkey@0"
I20260619 12:12:55.987118 17257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:56.040160 17257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:56.042371 17257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:56.042517 17257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:56.042793 17257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:56.043256 17313 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:12:56.045213 17257 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:12:56.045274 17257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 12:12:56.045316 17257 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:12:56.045919 17257 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:12:56.045975 17257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 12:12:56.046053 17313 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap starting.
I20260619 12:12:56.053049 17257 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:33261
I20260619 12:12:56.053128 17420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:33261 every 8 connection(s)
I20260619 12:12:56.053544 17257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:12:56.057964 17313 log.cc:824] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:12:56.060623 17313 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:12:56.060972 17313 tablet_bootstrap.cc:492] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Bootstrap complete.
I20260619 12:12:56.062244 17313 ts_tablet_manager.cc:1397] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260619 12:12:56.063581 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17257
I20260619 12:12:56.063845 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:36275
--local_ip_for_outbound_sockets=127.0.201.2
--tserver_master_addrs=127.0.201.62:38065
--webserver_port=39785
--webserver_interface=127.0.201.2
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:12:56.063635 17313 raft_consensus.cc:348] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:56.063946 17313 raft_consensus.cc:729] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc734ea500964d5bb3e0f8061c2a1c86, State: Initialized, Role: FOLLOWER
I20260619 12:12:56.064087 17313 consensus_queue.cc:260] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:56.064353 17313 ts_tablet_manager.cc:1428] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260619 12:12:56.096303 17423 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:12:56.054252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:43415)
0619 12:12:56.054426 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.054460 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:56.054898 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:56.096118 (+ 41220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
W20260619 12:12:56.096570 17421 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:38065 (0 consecutive failures): Network error: Failed to ping master at 127.0.201.62:38065: Client connection negotiation failed: client connection to 127.0.201.62:38065: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:12:56.096882 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.096768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:57131)
0619 12:12:56.096822 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.096865 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:12:56.097241 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.097160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:41787)
0619 12:12:56.097205 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.097226 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:12:56.097378 17421 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:12:56.148686 17426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:56.148878 17426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:56.148908 17426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:56.150059 17426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:56.150099 17426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:56.150151 17426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:12:56.151633 17426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:36275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=39785
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:56.151867 17426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:56.152093 17426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:56.160502 17434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:56.160539 17431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:56.160940 17426 server_base.cc:1034] running on GCE node
W20260619 12:12:56.161036 17432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:56.161283 17426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:56.161542 17426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:56.162709 17426 hybrid_clock.cc:648] HybridClock initialized: now 1781871176162682 us; error 40 us; skew 500 ppm
Jun 19 12:12:56 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:56.165884 17426 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:12:56.166324 17426 webserver.cc:460] Webserver started at http://127.0.201.2:39785/ using document root <none> and password file <none>
I20260619 12:12:56.166605 17426 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:56.166672 17426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:56.168154 17426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:12:56.169112 17441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:56.169363 17426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:12:56.169454 17426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "c660192d033542e5a7180a4463db3279"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "3bdd10e9e97d5a293a88d609a6b7e6c0"
server_key_iv: "d16dd4d468c4130c8f146986e426c968"
server_key_version: "encryptionkey@0"
I20260619 12:12:56.169891 17426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:56.185573 17426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:56.188186 17426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:56.188336 17426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:56.188565 17426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:56.188899 17426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:56.188939 17426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:56.188977 17426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:56.189002 17426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:56.195386 17426 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:36275
I20260619 12:12:56.195430 17554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:36275 every 8 connection(s)
I20260619 12:12:56.195777 17426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:12:56.196784 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.196430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:33027)
0619 12:12:56.196595 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.196658 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260619 12:12:56.197180 17555 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:38065 (0 consecutive failures): Network error: Failed to ping master at 127.0.201.62:38065: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
I20260619 12:12:56.197458 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.197363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:60849)
0619 12:12:56.197409 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.197440 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:12:56.197747 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.197662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:52793)
0619 12:12:56.197706 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.197728 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:12:56.197861 17555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:12:56.198727 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17426
I20260619 12:12:56.199025 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:39115
--local_ip_for_outbound_sockets=127.0.201.3
--tserver_master_addrs=127.0.201.62:38065
--webserver_port=35221
--webserver_interface=127.0.201.3
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:12:56.277478 17558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:12:56.277662 17558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:12:56.277684 17558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:12:56.278786 17558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:12:56.278825 17558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:12:56.278851 17558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:12:56.280459 17558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:39115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=35221
--tserver_master_addrs=127.0.201.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:12:56.280668 17558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:12:56.280884 17558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:12:56.285674 17564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:56.285663 17566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:12:56.285665 17563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:12:56.285931 17558 server_base.cc:1034] running on GCE node
I20260619 12:12:56.286111 17558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:12:56.286293 17558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:12:56.287452 17558 hybrid_clock.cc:648] HybridClock initialized: now 1781871176287441 us; error 26 us; skew 500 ppm
Jun 19 12:12:56 dist-test-slave-c910 krb5kdc[16121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:12:56.289839 17558 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:12:56.290266 17558 webserver.cc:460] Webserver started at http://127.0.201.3:35221/ using document root <none> and password file <none>
I20260619 12:12:56.290505 17558 fs_manager.cc:359] Metadata directory not provided
I20260619 12:12:56.290576 17558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:12:56.291940 17558 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:12:56.292493 17573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:56.292665 17558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:12:56.292761 17558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7"
format_stamp: "Formatted at 2026-06-19 12:12:14 on dist-test-slave-c910"
server_key: "f211fca897f841e3798ebb1a057b0d6d"
server_key_iv: "f8f523ca10009225399a1a3db2c2c4e1"
server_key_version: "encryptionkey@0"
I20260619 12:12:56.293081 17558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:12:56.307574 17558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:12:56.309233 17558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:12:56.309391 17558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:12:56.309602 17558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:12:56.309891 17558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:12:56.309952 17558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:56.309991 17558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:12:56.310021 17558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:12:56.316363 17558 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:39115
I20260619 12:12:56.316435 17686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:39115 every 8 connection(s)
I20260619 12:12:56.316754 17558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:12:56.317742 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.317389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:37401)
0619 12:12:56.317543 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.317608 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
W20260619 12:12:56.318161 17687 heartbeater.cc:644] Failed to heartbeat to 127.0.201.62:38065 (0 consecutive failures): Network error: Failed to ping master at 127.0.201.62:38065: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
I20260619 12:12:56.318444 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.318358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:38405)
0619 12:12:56.318405 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.318429 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:12:56.318697 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:56.318649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:34255)
0619 12:12:56.318668 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:56.318685 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 12:12:56.318814 17687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:12:56.324010 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17558
I20260619 12:12:57.097877 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.097666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:60667)
0619 12:12:57.097795 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:57.097830 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260619 12:12:57.198311 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.198125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:47515)
0619 12:12:57.198231 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:57.198264 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260619 12:12:57.319538 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.319146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:44025)
0619 12:12:57.319452 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:57.319486 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":205,"threads_started":1}
I20260619 12:12:57.447083 17694 raft_consensus.cc:482] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:57.447268 17694 raft_consensus.cc:504] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:57.447578 17694 leader_election.cc:288] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), c660192d033542e5a7180a4463db3279 (127.0.201.2:36275)
Jun 19 12:12:57 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.2@KRBTEST.COM
Jun 19 12:12:57 dist-test-slave-c910 krb5kdc[16121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.3@KRBTEST.COM
I20260619 12:12:57.458212 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.447726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:39115 (local address 127.0.201.1:37395)
0619 12:12:57.447766 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:57.447778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:57.447839 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:57.448739 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:57.448742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:57.449118 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:57.449292 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:57.449298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.450385 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.450387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:57.451152 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:57.451154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.451804 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.451805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:57.451915 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:57.452643 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:57.452647 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:57.454535 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:57.456539 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:57.456540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:57.456540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:57.456801 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:57.457024 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:57.457025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:57.457025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:57.457178 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:57.457621 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:57.457627 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:57.457789 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 12:12:57.457939 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 12:12:57.458099 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":87}
I20260619 12:12:57.458225 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.447861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:37395 (local address 127.0.201.3:39115)
0619 12:12:57.448031 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:57.448044 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:57.448078 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:12:57.448517 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:57.448518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:57.448556 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:57.448708 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:57.449414 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.450270 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.451306 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.451708 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.451829 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:57.454653 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:57.454657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:57.454657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:57.454671 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:57.456472 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:57.456869 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:57.456872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:57.456872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:57.456955 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:57.457252 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:57.457253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:57.457253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:57.457543 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:57.457627 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:57.457996 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 12:12:57.458116 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":57,"threads_started":1}
I20260619 12:12:57.458225 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.447889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:46705 (local address 127.0.201.2:36275)
0619 12:12:57.448054 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 12:12:57.448062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:12:57.448089 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:12:57.448513 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:12:57.448515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:12:57.448556 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:12:57.448710 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:12:57.449435 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.450271 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.451280 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.451707 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.451841 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:57.454164 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:12:57.454169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:12:57.454169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:12:57.454181 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:12:57.456289 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:57.456815 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:57.456817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:57.456818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:57.456945 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:12:57.457252 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:12:57.457253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:12:57.457253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:12:57.457550 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:12:57.457628 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:12:57.457987 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 12:12:57.458116 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260619 12:12:57.458263 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:57.447771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36275 (local address 127.0.201.1:46705)
0619 12:12:57.447893 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:57.447904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:12:57.447956 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:12:57.448741 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:12:57.448743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:12:57.449120 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:12:57.449292 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:57.449298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.450381 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.450382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:57.451151 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:12:57.451153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:12:57.451802 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:12:57.451803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:12:57.451913 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:12:57.452585 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:12:57.452595 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:12:57.454061 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:12:57.456370 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:57.456384 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:57.456386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:57.456733 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:57.457024 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:12:57.457025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:12:57.457025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:12:57.457178 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:12:57.457621 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:12:57.457627 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:12:57.457789 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 12:12:57.457913 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 12:12:57.458151 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":113,"thread_start_us":56,"threads_started":1}
W20260619 12:12:57.459371 17310 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c660192d033542e5a7180a4463db3279 (127.0.201.2:36275): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
W20260619 12:12:57.459467 17307 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
I20260619 12:12:57.459537 17307 leader_election.cc:302] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc734ea500964d5bb3e0f8061c2a1c86; no voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279
I20260619 12:12:57.459635 17694 raft_consensus.cc:2726] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:12:58.098495 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:58.098292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:36019)
0619 12:12:58.098420 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:58.098451 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260619 12:12:58.198841 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:58.198672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:46581)
0619 12:12:58.198776 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:58.198806 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260619 12:12:58.320163 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:58.319969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:45285)
0619 12:12:58.320087 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:58.320124 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260619 12:12:59.038748 17701 raft_consensus.cc:482] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:12:59.038921 17701 raft_consensus.cc:504] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:12:59.039145 17701 leader_election.cc:288] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), c660192d033542e5a7180a4463db3279 (127.0.201.2:36275)
W20260619 12:12:59.039588 17307 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
W20260619 12:12:59.039683 17310 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c660192d033542e5a7180a4463db3279 (127.0.201.2:36275): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
I20260619 12:12:59.039759 17310 leader_election.cc:302] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc734ea500964d5bb3e0f8061c2a1c86; no voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279
I20260619 12:12:59.039831 17701 raft_consensus.cc:2726] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:12:59.099128 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:59.098921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:43861)
0619 12:12:59.099049 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:59.099083 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260619 12:12:59.199549 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:59.199320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:36637)
0619 12:12:59.199477 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:59.199512 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260619 12:12:59.320734 17704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:12:59.320523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:57823)
0619 12:12:59.320664 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:12:59.320699 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260619 12:13:00.099759 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:00.099560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:38497)
0619 12:13:00.099672 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:00.099713 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260619 12:13:00.200245 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:00.200050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:38549)
0619 12:13:00.200172 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:00.200208 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260619 12:13:00.321419 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:00.321214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:43263)
0619 12:13:00.321349 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:00.321381 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260619 12:13:01.100405 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:01.100197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:38103)
0619 12:13:01.100318 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:01.100353 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260619 12:13:01.200898 17709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:01.200684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:58447)
0619 12:13:01.200821 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:01.200858 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260619 12:13:01.322079 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:01.321889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:42379)
0619 12:13:01.322008 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:01.322040 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260619 12:13:01.783447 17711 raft_consensus.cc:482] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:01.783533 17711 raft_consensus.cc:504] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:13:01.783751 17711 leader_election.cc:288] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), c660192d033542e5a7180a4463db3279 (127.0.201.2:36275)
W20260619 12:13:01.784081 17310 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c660192d033542e5a7180a4463db3279 (127.0.201.2:36275): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
W20260619 12:13:01.784173 17307 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
I20260619 12:13:01.784255 17307 leader_election.cc:302] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc734ea500964d5bb3e0f8061c2a1c86; no voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279
I20260619 12:13:01.784333 17711 raft_consensus.cc:2726] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:13:02.101140 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:02.100899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:33689)
0619 12:13:02.101052 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:02.101091 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":60,"threads_started":1}
I20260619 12:13:02.201506 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:02.201327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:55113)
0619 12:13:02.201437 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:02.201470 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260619 12:13:02.322764 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:02.322558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:46223)
0619 12:13:02.322674 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:02.322707 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260619 12:13:03.101792 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:03.101593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:50677)
0619 12:13:03.101715 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:03.101745 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260619 12:13:03.202204 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:03.202004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:46967)
0619 12:13:03.202129 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:03.202164 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260619 12:13:03.323521 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:03.323261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:43515)
0619 12:13:03.323429 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:03.323470 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260619 12:13:04.102438 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:04.102227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:50235)
0619 12:13:04.102359 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:04.102392 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260619 12:13:04.202853 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:04.202648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:50895)
0619 12:13:04.202769 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:04.202811 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260619 12:13:04.324211 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:04.323991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:53177)
0619 12:13:04.324135 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:04.324170 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260619 12:13:05.103067 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:05.102865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:50881)
0619 12:13:05.102982 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:05.103015 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260619 12:13:05.203613 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:05.203408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:50525)
0619 12:13:05.203533 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:05.203569 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260619 12:13:05.324841 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:05.324658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:51857)
0619 12:13:05.324769 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:05.324802 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260619 12:13:06.103709 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:06.103522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:49679)
0619 12:13:06.103636 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:06.103666 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260619 12:13:06.204310 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:06.204092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:52193)
0619 12:13:06.204236 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:06.204269 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260619 12:13:06.325480 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:06.325274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:44583)
0619 12:13:06.325406 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:06.325440 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260619 12:13:06.530953 17727 raft_consensus.cc:482] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:06.531080 17727 raft_consensus.cc:504] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bc6ac03f8d1449ebf5aeb2a3654bfa7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 39115 } } peers { permanent_uuid: "c660192d033542e5a7180a4463db3279" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36275 } } peers { permanent_uuid: "fc734ea500964d5bb3e0f8061c2a1c86" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 33261 } }
I20260619 12:13:06.531323 17727 leader_election.cc:288] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115), c660192d033542e5a7180a4463db3279 (127.0.201.2:36275)
W20260619 12:13:06.531659 17307 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bc6ac03f8d1449ebf5aeb2a3654bfa7 (127.0.201.3:39115): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
W20260619 12:13:06.531759 17310 leader_election.cc:341] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c660192d033542e5a7180a4463db3279 (127.0.201.2:36275): Not found: Tablet not found: 4215b91e72c1434498430912d7655bb8
I20260619 12:13:06.531844 17310 leader_election.cc:302] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc734ea500964d5bb3e0f8061c2a1c86; no voters: 5bc6ac03f8d1449ebf5aeb2a3654bfa7, c660192d033542e5a7180a4463db3279
I20260619 12:13:06.531932 17727 raft_consensus.cc:2726] T 4215b91e72c1434498430912d7655bb8 P fc734ea500964d5bb3e0f8061c2a1c86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:13:07.104339 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:07.104125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:37293)
0619 12:13:07.104255 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:07.104288 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260619 12:13:07.205070 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:07.204886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:49951)
0619 12:13:07.204995 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:07.205029 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260619 12:13:07.326192 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:07.325973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:46139)
0619 12:13:07.326096 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:07.326147 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260619 12:13:08.104964 17731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:08.104742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:56495)
0619 12:13:08.104880 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:08.104915 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":56,"threads_started":1}
I20260619 12:13:08.205708 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:08.205514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:45219)
0619 12:13:08.205634 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:08.205669 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260619 12:13:08.326859 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:08.326607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:36427)
0619 12:13:08.326780 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:08.326816 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":59,"threads_started":1}
I20260619 12:13:09.105746 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:09.105440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:56785)
0619 12:13:09.105604 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:09.105658 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
I20260619 12:13:09.206288 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:09.206104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:45067)
0619 12:13:09.206220 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:09.206251 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260619 12:13:09.327554 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:09.327351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:54993)
0619 12:13:09.327472 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:09.327509 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260619 12:13:10.106563 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:10.106340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:40771)
0619 12:13:10.106474 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:10.106513 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260619 12:13:10.206915 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:10.206666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:44499)
0619 12:13:10.206796 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:10.206872 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260619 12:13:10.328219 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:10.327966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:48411)
0619 12:13:10.328100 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:10.328161 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260619 12:13:11.107558 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:11.107329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.1:44217)
0619 12:13:11.107475 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:11.107511 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260619 12:13:11.207651 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:11.207449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.2:35871)
0619 12:13:11.207574 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:11.207606 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260619 12:13:11.328989 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:11.328758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38065 (local address 127.0.201.3:39333)
0619 12:13:11.328902 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:11.328940 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.62:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260619 12:13:11.332751 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17257
I20260619 12:13:11.337519 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17426
I20260619 12:13:11.341802 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17558
2026-06-19T12:13:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58314 ms)
[----------] 30 tests from SecurityITest (228182 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:11 dist-test-slave-c910 krb5kdc[17745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:11 dist-test-slave-c910 krb5kdc[17745](info): set up 2 sockets
Jun 19 12:13:11 dist-test-slave-c910 krb5kdc[17745](info): commencing operation
krb5kdc: starting...
W20260619 12:13:12.375001 804 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:12.494091 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45803
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:45803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:12.575990 17758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:12.576223 17758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:12.576247 17758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:12.577422 17758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:12.577467 17758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:12.577482 17758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:12.577495 17758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:12.577530 17758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:12.579066 17758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:45803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:12.579350 17758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:12.579587 17758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:12.584837 17764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:12.584851 17763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:12.584851 17766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:12.584877 17758 server_base.cc:1034] running on GCE node
I20260619 12:13:12.585279 17758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:12.585538 17758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:12.586715 17758 hybrid_clock.cc:648] HybridClock initialized: now 1781871192586701 us; error 31 us; skew 500 ppm
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:12.589236 17758 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:12.589704 17758 webserver.cc:460] Webserver started at http://127.0.201.62:44947/ using document root <none> and password file <none>
I20260619 12:13:12.589936 17758 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:12.590004 17758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:12.590154 17758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:12.591291 17758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "1851b4649cd8475b8252abc50745d01f"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "72b08af76e9f67cf50397f121f1b9297"
server_key_iv: "0df90c32675810913756d37f38fd0879"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.591663 17758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "1851b4649cd8475b8252abc50745d01f"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "72b08af76e9f67cf50397f121f1b9297"
server_key_iv: "0df90c32675810913756d37f38fd0879"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.593025 17758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:13:12.593781 17773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.593993 17758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:13:12.594101 17758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "1851b4649cd8475b8252abc50745d01f"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "72b08af76e9f67cf50397f121f1b9297"
server_key_iv: "0df90c32675810913756d37f38fd0879"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.594175 17758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:12.612123 17758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:12.614567 17758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:12.614737 17758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:12.618896 17758 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:45803
I20260619 12:13:12.618919 17825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:45803 every 8 connection(s)
I20260619 12:13:12.619387 17758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:12.620364 17826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:12.622637 17826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f: Bootstrap starting.
I20260619 12:13:12.623178 17826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:12.623443 17826 log.cc:824] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:12.624143 17826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f: No bootstrap required, opened a new log
I20260619 12:13:12.625655 17826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1851b4649cd8475b8252abc50745d01f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45803 } }
I20260619 12:13:12.625793 17826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:12.625824 17826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1851b4649cd8475b8252abc50745d01f, State: Initialized, Role: FOLLOWER
I20260619 12:13:12.625936 17826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1851b4649cd8475b8252abc50745d01f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45803 } }
I20260619 12:13:12.626032 17826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:12.626066 17826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:12.626124 17826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:12.626672 17826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1851b4649cd8475b8252abc50745d01f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45803 } }
I20260619 12:13:12.626793 17826 leader_election.cc:302] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1851b4649cd8475b8252abc50745d01f; no voters:
I20260619 12:13:12.626984 17826 leader_election.cc:288] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:12.627028 17829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:12.627220 17829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [term 1 LEADER]: Becoming Leader. State: Replica: 1851b4649cd8475b8252abc50745d01f, State: Running, Role: LEADER
I20260619 12:13:12.627277 17826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:12.627343 17829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1851b4649cd8475b8252abc50745d01f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45803 } }
I20260619 12:13:12.627892 17830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1851b4649cd8475b8252abc50745d01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1851b4649cd8475b8252abc50745d01f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45803 } } }
I20260619 12:13:12.627985 17830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:12.628072 17831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1851b4649cd8475b8252abc50745d01f. Latest consensus state: current_term: 1 leader_uuid: "1851b4649cd8475b8252abc50745d01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1851b4649cd8475b8252abc50745d01f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45803 } } }
I20260619 12:13:12.628309 17831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:12.628358 17838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:12.628902 17838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:12.629185 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17758
I20260619 12:13:12.629304 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:12.629521 804 external_mini_cluster.cc:1428] Setting key 589aa0dd44b54de57a1355383531b8bd
I20260619 12:13:12.630995 17838 catalog_manager.cc:1347] Generated new cluster ID: d0dc40287ea242f59baa2e3d041dd2a9
I20260619 12:13:12.631047 17838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:12.641448 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:12.630001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.0.201.62:45803)
0619 12:13:12.630315 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:12.630327 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:12.630361 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:13:12.630479 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:12.630484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:12.630526 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:12.630634 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:12.631468 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:12.632853 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:12.633977 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:12.634580 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:12.634673 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:12.637676 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:12.637681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:12.637681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:12.637694 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:12.639689 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:12.640105 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:12.640108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:12.640108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:12.640213 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:12.640424 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:12.640425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:12.640425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:12.640712 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:12.640852 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:12.640980 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 12:13:12.641213 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":68,"threads_started":1}
I20260619 12:13:12.651015 17838 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:12.651806 17838 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:12.667376 17838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f: Generated new TSK 0
I20260619 12:13:12.667623 17838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:13:12.694692 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45803
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:12.787420 17852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:12.787632 17852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:12.787659 17852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:12.788791 17852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:12.788828 17852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:12.788854 17852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:13:12.790362 17852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:12.790565 17852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:12.790753 17852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:12.795881 17857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:12.795883 17858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:12.795883 17860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:12.796171 17852 server_base.cc:1034] running on GCE node
I20260619 12:13:12.796393 17852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:12.796597 17852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:12.797739 17852 hybrid_clock.cc:648] HybridClock initialized: now 1781871192797691 us; error 66 us; skew 500 ppm
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:12.800480 17852 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:13:12.801002 17852 webserver.cc:460] Webserver started at http://127.0.201.1:39887/ using document root <none> and password file <none>
I20260619 12:13:12.801254 17852 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:12.801324 17852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:12.801476 17852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:12.802552 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "d8b78c8f3e8441e69c65fa20577c4fe7"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "5d21b8b71de80e5de3d72f3e873b5d47"
server_key_iv: "027e32a42def467b1526b7a61ba7e91e"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.802928 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "d8b78c8f3e8441e69c65fa20577c4fe7"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "5d21b8b71de80e5de3d72f3e873b5d47"
server_key_iv: "027e32a42def467b1526b7a61ba7e91e"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.804401 17852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:12.805250 17867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.805450 17852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:12.805546 17852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "d8b78c8f3e8441e69c65fa20577c4fe7"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "5d21b8b71de80e5de3d72f3e873b5d47"
server_key_iv: "027e32a42def467b1526b7a61ba7e91e"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.805634 17852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:12.830552 17852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:12.833204 17852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:12.833396 17852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:12.833659 17852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:12.833981 17852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:12.834036 17852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.834077 17852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:12.834105 17852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.840111 17852 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:45791
I20260619 12:13:12.840155 17980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:45791 every 8 connection(s)
I20260619 12:13:12.840529 17852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:12.850131 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17852
I20260619 12:13:12.850261 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:13:12.850361 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:12.841345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:33785 (local address 127.0.201.62:45803)
0619 12:13:12.841414 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:12.841416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:12.841590 (+ 174us) server_negotiation.cc:407] Connection header received
0619 12:13:12.842594 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:12.842596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:12.842611 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:12.842670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:12.843347 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:12.843873 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:12.844908 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:12.845138 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:12.845226 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:12.847516 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:12.847517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:12.847517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:12.847528 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:12.848797 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:12.849262 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:12.849263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:12.849264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:12.849325 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:12.849572 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:12.849574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:12.849574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:12.849696 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:12.849771 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:12.850115 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 12:13:12.850235 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:13:12.850477 804 external_mini_cluster.cc:1428] Setting key 770b929d37c22477c9fd0514ad11776d
I20260619 12:13:12.850515 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:12.841290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45803 (local address 127.0.201.1:33785)
0619 12:13:12.841463 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:12.841500 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:12.842470 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:12.842749 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:12.842753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:12.843019 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:12.843238 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:12.843242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:12.843974 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:12.843976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:12.844805 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:12.844807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:12.845238 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:12.845239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:12.845374 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:12.845948 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:12.845956 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:12.847411 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:12.848948 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:12.848956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:12.848958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:12.849182 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:12.849395 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:12.849396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:12.849396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:12.849504 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:12.849763 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:12.849769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:12.849932 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 12:13:12.850060 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 12:13:12.850256 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260619 12:13:12.851168 17981 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45803
I20260619 12:13:12.851315 17981 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:12.851503 17981 heartbeater.cc:511] Master 127.0.201.62:45803 requested a full tablet report, sending...
I20260619 12:13:12.852108 17790 ts_manager.cc:194] Registered new tserver with Master: d8b78c8f3e8441e69c65fa20577c4fe7 (127.0.201.1:45791)
I20260619 12:13:12.853010 17790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:33785
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:13:12.877785 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45803
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:12.959587 17986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:12.959800 17986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:12.959824 17986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:12.960923 17986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:12.960958 17986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:12.960984 17986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:13:12.962386 17986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:12.962574 17986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:12.962760 17986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:12.967895 17991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:12.967907 17992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:12.967960 17994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:12.968091 17986 server_base.cc:1034] running on GCE node
I20260619 12:13:12.968323 17986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:12.968585 17986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:12.969754 17986 hybrid_clock.cc:648] HybridClock initialized: now 1781871192969705 us; error 66 us; skew 500 ppm
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:12.972349 17986 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:13:12.972800 17986 webserver.cc:460] Webserver started at http://127.0.201.2:44111/ using document root <none> and password file <none>
I20260619 12:13:12.973031 17986 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:12.973101 17986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:12.973253 17986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:12.974318 17986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "f2c9e7ffa088464db3ce4c150cfe71e6"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "19dc3f2ebfc0c263370828c805fb399f"
server_key_iv: "e0bc2a9dfd025bf961813255a69580f5"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.974740 17986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "f2c9e7ffa088464db3ce4c150cfe71e6"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "19dc3f2ebfc0c263370828c805fb399f"
server_key_iv: "e0bc2a9dfd025bf961813255a69580f5"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.976198 17986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:12.977015 18001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.977201 17986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:12.977308 17986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "f2c9e7ffa088464db3ce4c150cfe71e6"
format_stamp: "Formatted at 2026-06-19 12:13:12 on dist-test-slave-c910"
server_key: "19dc3f2ebfc0c263370828c805fb399f"
server_key_iv: "e0bc2a9dfd025bf961813255a69580f5"
server_key_version: "encryptionkey@0"
I20260619 12:13:12.977398 17986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:12.988857 17986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:12.990923 17986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:12.991097 17986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:12.991379 17986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:12.991708 17986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:12.991770 17986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.991806 17986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:12.991874 17986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:12.998036 17986 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:44595
I20260619 12:13:12.998076 18114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:44595 every 8 connection(s)
I20260619 12:13:12.998420 17986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:13:13.002812 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 17986
I20260619 12:13:13.002931 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:13:13.003127 804 external_mini_cluster.cc:1428] Setting key 33f6150495eae8491d2202e22fd113b5
Jun 19 12:13:12 dist-test-slave-c910 krb5kdc[17745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:13.008985 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:12.999162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:38047 (local address 127.0.201.62:45803)
0619 12:13:12.999261 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:12.999263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:12.999472 (+ 209us) server_negotiation.cc:407] Connection header received
0619 12:13:13.000447 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.000449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.000462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.000515 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.001204 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.001714 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.002752 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.003055 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.003133 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.005645 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.005647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.005647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:13.005656 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.007322 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:13.007861 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:13.007862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:13.007863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:13.007916 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:13.008202 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:13.008203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:13.008203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:13.008322 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.008372 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.008841 (+ 469us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.008917 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:13:13.009254 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:12.999146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45803 (local address 127.0.201.2:38047)
0619 12:13:12.999347 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:12.999383 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.000329 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.000604 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.000608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.000885 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:13.001092 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.001096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.001807 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.001808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.002646 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.002648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.003137 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.003138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.003308 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.003879 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:13.003888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:13.005544 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:13.007404 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:13.007412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:13.007414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:13.007771 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:13.007983 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:13.007984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:13.007984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:13.008135 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:13.008390 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:13.008397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:13.008590 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.008747 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.008955 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
I20260619 12:13:13.009770 18115 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45803
I20260619 12:13:13.009894 18115 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:13.010083 18115 heartbeater.cc:511] Master 127.0.201.62:45803 requested a full tablet report, sending...
I20260619 12:13:13.010546 17790 ts_manager.cc:194] Registered new tserver with Master: f2c9e7ffa088464db3ce4c150cfe71e6 (127.0.201.2:44595)
I20260619 12:13:13.010972 17790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:38047
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:13:13.029670 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45803
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:13.112447 18120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:13.112679 18120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:13.112703 18120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:13.113826 18120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:13.113873 18120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:13.113898 18120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:13:13.115774 18120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:13.116036 18120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:13.116271 18120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:13.121438 18128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.121507 18126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.121428 18125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:13.121592 18120 server_base.cc:1034] running on GCE node
I20260619 12:13:13.121819 18120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:13.122082 18120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:13.123255 18120 hybrid_clock.cc:648] HybridClock initialized: now 1781871193123229 us; error 41 us; skew 500 ppm
Jun 19 12:13:13 dist-test-slave-c910 krb5kdc[17745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:13.125758 18120 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:13:13.126190 18120 webserver.cc:460] Webserver started at http://127.0.201.3:42029/ using document root <none> and password file <none>
I20260619 12:13:13.126434 18120 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:13.126479 18120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:13.126646 18120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:13.127887 18120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "f6682c5cea854ad7bb4774acbd4efacf"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "b601f19e687ffbd25637b64afecf5082"
server_key_iv: "89320714719793d6f89b0449af2dd34f"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.128302 18120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "f6682c5cea854ad7bb4774acbd4efacf"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "b601f19e687ffbd25637b64afecf5082"
server_key_iv: "89320714719793d6f89b0449af2dd34f"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.129701 18120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:13:13.130465 18135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.130692 18120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:13.130790 18120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "f6682c5cea854ad7bb4774acbd4efacf"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "b601f19e687ffbd25637b64afecf5082"
server_key_iv: "89320714719793d6f89b0449af2dd34f"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.130872 18120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:13.167789 18120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:13.169948 18120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:13.170120 18120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:13.170346 18120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:13.170701 18120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:13.170733 18120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.170778 18120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:13.170804 18120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.176606 18120 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:45093
I20260619 12:13:13.176692 18248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:45093 every 8 connection(s)
I20260619 12:13:13.177021 18120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:13:13 dist-test-slave-c910 krb5kdc[17745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:13.185084 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18120
I20260619 12:13:13.185160 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:13:13.185359 804 external_mini_cluster.cc:1428] Setting key 9c2bdbb44255d1f87c1d9c60d4e57aa8
I20260619 12:13:13.186890 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.177768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:36183 (local address 127.0.201.62:45803)
0619 12:13:13.177855 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.177856 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.178019 (+ 163us) server_negotiation.cc:407] Connection header received
0619 12:13:13.178987 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.178989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.179002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.179059 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.179826 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.180415 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.181549 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.181751 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.181840 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.184026 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.184027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.184028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:13.184035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.185369 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:13.185978 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:13.185979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:13.185979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:13.186041 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:13.186322 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:13.186323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:13.186323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:13.186462 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.186515 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.186760 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.186828 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:13:13.187067 18251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.177742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45803 (local address 127.0.201.3:36183)
0619 12:13:13.177893 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.177937 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.178861 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.179145 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.179148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.179481 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:13.179727 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.179732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.180506 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.180507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.181462 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.181464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.181864 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.181865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.181989 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.182484 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:13.182498 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:13.183921 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:13.185467 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:13.185475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:13.185477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:13.185843 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:13.186107 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:13.186108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:13.186109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:13.186254 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:13.186524 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:13.186528 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:13.186647 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.186732 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.186851 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260619 12:13:13.187582 18249 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45803
I20260619 12:13:13.187681 18249 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:13.187889 18249 heartbeater.cc:511] Master 127.0.201.62:45803 requested a full tablet report, sending...
I20260619 12:13:13.188256 17790 ts_manager.cc:194] Registered new tserver with Master: f6682c5cea854ad7bb4774acbd4efacf (127.0.201.3:45093)
I20260619 12:13:13.188624 17790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:36183
I20260619 12:13:13.197248 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:13:13.204566 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.198607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.0.201.62:45803)
0619 12:13:13.198651 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.198653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.198785 (+ 132us) server_negotiation.cc:407] Connection header received
0619 12:13:13.198841 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.198841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.198849 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.198899 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.199536 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.200019 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.200727 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.200904 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.200953 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.202111 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.202112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.202112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:13.202119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.203666 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:13.203956 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:13.203957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:13.203957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:13.204005 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:13.204145 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:13.204145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:13.204146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:13.204292 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.204364 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.204444 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.204501 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:13:13.206260 17790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:13:13.208004 17790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:13:13.214967 18260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.210472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:45791 (local address 127.0.0.1:46838)
0619 12:13:13.210823 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.210834 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.210886 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.211508 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.211509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.211511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.211555 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.211556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.212710 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.212711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.213784 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.213787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.214657 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.214657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.214734 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.214749 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.214827 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.214877 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":46,"thread_start_us":54,"threads_started":1}
I20260619 12:13:13.214985 18263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.210670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46838 (local address 127.0.201.1:45791)
0619 12:13:13.211359 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.211364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.211374 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:13:13.211411 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.211413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.211427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.211511 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.211595 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.212615 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.213884 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.214572 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.214649 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.214707 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.214855 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.214905 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":74,"threads_started":1}
I20260619 12:13:13.215301 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.210604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.0.201.2:44595)
0619 12:13:13.210800 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.210806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.210818 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:13:13.211091 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.211094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.211114 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.211215 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.211400 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.212581 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.214114 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.214850 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.214914 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.214964 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.215166 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.215229 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260619 12:13:13.215530 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.210470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:44595 (local address 127.0.0.1:36076)
0619 12:13:13.210650 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.210692 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.210760 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.211222 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.211226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.211228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.211306 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.211308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.212707 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.212708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.214020 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.214022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.214999 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.215000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.215064 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.215069 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.215394 (+ 325us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.215445 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":39,"thread_start_us":104,"threads_started":1}
I20260619 12:13:13.215951 17915 tablet_service.cc:1467] Processing CreateTablet for tablet 93c8fa2bde594a79ada7558ee0b400ce (DEFAULT_TABLE table=test-table [id=b622c2a6996b48c589324dccac4df78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:13.216162 18049 tablet_service.cc:1467] Processing CreateTablet for tablet 93c8fa2bde594a79ada7558ee0b400ce (DEFAULT_TABLE table=test-table [id=b622c2a6996b48c589324dccac4df78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:13.216317 17915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93c8fa2bde594a79ada7558ee0b400ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.216461 18049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93c8fa2bde594a79ada7558ee0b400ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.218937 18266 tablet_bootstrap.cc:492] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: Bootstrap starting.
I20260619 12:13:13.218947 18267 tablet_bootstrap.cc:492] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Bootstrap starting.
I20260619 12:13:13.219585 18267 tablet_bootstrap.cc:654] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.219630 18266 tablet_bootstrap.cc:654] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.220098 18267 log.cc:824] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.220161 18266 log.cc:824] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.220779 18267 tablet_bootstrap.cc:492] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: No bootstrap required, opened a new log
I20260619 12:13:13.220866 18267 ts_tablet_manager.cc:1397] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:13.220872 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.210690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45093 (local address 127.0.0.1:41850)
0619 12:13:13.212743 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.212753 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.212799 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.213079 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.213080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.213081 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.213115 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.213117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.214397 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.214398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.219605 (+ 5207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.219609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.220641 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.220642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.220688 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.220691 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.220759 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.220801 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2034,"thread_start_us":78,"threads_started":1}
I20260619 12:13:13.220891 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.210897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.0.201.3:45093)
0619 12:13:13.211086 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.211089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.212853 (+ 1764us) server_negotiation.cc:407] Connection header received
0619 12:13:13.212901 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.212903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.212929 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.212996 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.213188 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.214188 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.219725 (+ 5537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.220517 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.220556 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.220591 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.220775 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.220811 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260619 12:13:13.221464 18183 tablet_service.cc:1467] Processing CreateTablet for tablet 93c8fa2bde594a79ada7558ee0b400ce (DEFAULT_TABLE table=test-table [id=b622c2a6996b48c589324dccac4df78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:13.221558 18266 tablet_bootstrap.cc:492] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: No bootstrap required, opened a new log
I20260619 12:13:13.221632 18266 ts_tablet_manager.cc:1397] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:13:13.221776 18183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93c8fa2bde594a79ada7558ee0b400ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.223039 18267 raft_consensus.cc:348] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.223258 18267 raft_consensus.cc:374] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.223302 18267 raft_consensus.cc:729] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c9e7ffa088464db3ce4c150cfe71e6, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.223426 18267 consensus_queue.cc:260] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.223691 18267 ts_tablet_manager.cc:1428] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 12:13:13.223687 18266 raft_consensus.cc:348] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.223829 18115 heartbeater.cc:503] Master 127.0.201.62:45803 was elected leader, sending a full tablet report...
I20260619 12:13:13.223855 18266 raft_consensus.cc:374] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.223903 18266 raft_consensus.cc:729] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b78c8f3e8441e69c65fa20577c4fe7, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.224011 18266 consensus_queue.cc:260] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.224269 18266 ts_tablet_manager.cc:1428] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:13.224427 17981 heartbeater.cc:503] Master 127.0.201.62:45803 was elected leader, sending a full tablet report...
I20260619 12:13:13.224684 18272 tablet_bootstrap.cc:492] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Bootstrap starting.
I20260619 12:13:13.225304 18272 tablet_bootstrap.cc:654] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.225790 18272 log.cc:824] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.226568 18272 tablet_bootstrap.cc:492] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: No bootstrap required, opened a new log
I20260619 12:13:13.226620 18272 ts_tablet_manager.cc:1397] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:13.228071 18272 raft_consensus.cc:348] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.228197 18272 raft_consensus.cc:374] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.228219 18272 raft_consensus.cc:729] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6682c5cea854ad7bb4774acbd4efacf, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.228317 18272 consensus_queue.cc:260] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.228575 18272 ts_tablet_manager.cc:1428] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:13:13.228621 18249 heartbeater.cc:503] Master 127.0.201.62:45803 was elected leader, sending a full tablet report...
W20260619 12:13:13.249114 18116 tablet.cc:2369] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:13:13.295071 18270 raft_consensus.cc:482] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:13.295225 18270 raft_consensus.cc:504] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.295650 18270 leader_election.cc:288] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8b78c8f3e8441e69c65fa20577c4fe7 (127.0.201.1:45791), f6682c5cea854ad7bb4774acbd4efacf (127.0.201.3:45093)
I20260619 12:13:13.299772 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.296085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:59343 (local address 127.0.201.3:45093)
0619 12:13:13.296146 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.296148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.296245 (+ 97us) server_negotiation.cc:407] Connection header received
0619 12:13:13.296295 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.296296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.296306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.296354 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.296504 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.297183 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.298742 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.299340 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.299424 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.299433 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.299660 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.299699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:13:13.299772 18263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.295946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:39465 (local address 127.0.201.1:45791)
0619 12:13:13.296037 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.296039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.296048 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:13.296084 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.296085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.296094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.296186 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.296414 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.297184 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.298742 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.299340 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.299416 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.299428 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.299660 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.299699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 12:13:13.299783 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.295842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:45791 (local address 127.0.201.2:39465)
0619 12:13:13.295904 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.295918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.295965 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.296205 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.296206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.296208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.296312 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.296315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.297305 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.297306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.298650 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.298653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.299451 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.299452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.299559 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.299561 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.299636 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.299692 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":45}
I20260619 12:13:13.299793 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.295977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:45093 (local address 127.0.201.2:59343)
0619 12:13:13.296172 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.296183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.296241 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.296378 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.296379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.296381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.296444 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.296445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.297305 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.297306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.298650 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.298653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.299451 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.299452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.299557 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.299562 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.299636 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.299691 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":85,"thread_start_us":124,"threads_started":1}
I20260619 12:13:13.300180 17935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93c8fa2bde594a79ada7558ee0b400ce" candidate_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" is_pre_election: true
I20260619 12:13:13.300369 17935 raft_consensus.cc:2454] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c9e7ffa088464db3ce4c150cfe71e6 in term 0.
I20260619 12:13:13.300580 18203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93c8fa2bde594a79ada7558ee0b400ce" candidate_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6682c5cea854ad7bb4774acbd4efacf" is_pre_election: true
I20260619 12:13:13.300683 18005 leader_election.cc:302] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8b78c8f3e8441e69c65fa20577c4fe7, f2c9e7ffa088464db3ce4c150cfe71e6; no voters:
I20260619 12:13:13.300714 18203 raft_consensus.cc:2454] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c9e7ffa088464db3ce4c150cfe71e6 in term 0.
I20260619 12:13:13.300819 18270 raft_consensus.cc:2781] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:13:13.300933 18270 raft_consensus.cc:482] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:13:13.300973 18270 raft_consensus.cc:3037] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:13.301676 18270 raft_consensus.cc:504] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.301801 18270 leader_election.cc:288] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [CANDIDATE]: Term 1 election: Requested vote from peers d8b78c8f3e8441e69c65fa20577c4fe7 (127.0.201.1:45791), f6682c5cea854ad7bb4774acbd4efacf (127.0.201.3:45093)
I20260619 12:13:13.301949 17935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93c8fa2bde594a79ada7558ee0b400ce" candidate_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7"
I20260619 12:13:13.301966 18203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93c8fa2bde594a79ada7558ee0b400ce" candidate_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6682c5cea854ad7bb4774acbd4efacf"
I20260619 12:13:13.302037 18203 raft_consensus.cc:3037] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:13.302030 17935 raft_consensus.cc:3037] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:13.302771 18203 raft_consensus.cc:2454] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c9e7ffa088464db3ce4c150cfe71e6 in term 1.
I20260619 12:13:13.302776 17935 raft_consensus.cc:2454] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c9e7ffa088464db3ce4c150cfe71e6 in term 1.
I20260619 12:13:13.302932 18005 leader_election.cc:302] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f2c9e7ffa088464db3ce4c150cfe71e6, f6682c5cea854ad7bb4774acbd4efacf; no voters:
I20260619 12:13:13.303035 18270 raft_consensus.cc:2781] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:13.303241 18270 raft_consensus.cc:686] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 1 LEADER]: Becoming Leader. State: Replica: f2c9e7ffa088464db3ce4c150cfe71e6, State: Running, Role: LEADER
I20260619 12:13:13.303354 18270 consensus_queue.cc:237] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } }
I20260619 12:13:13.304171 17790 catalog_manager.cc:5495] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c9e7ffa088464db3ce4c150cfe71e6 (127.0.201.2). New cstate: current_term: 1 leader_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2c9e7ffa088464db3ce4c150cfe71e6" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 44595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 } health_report { overall_health: UNKNOWN } } }
I20260619 12:13:13.330194 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.327557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.0.201.2:44595)
0619 12:13:13.327628 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.327630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.327639 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:13.327684 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.327690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.327711 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.327822 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:13.327975 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.328534 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.329342 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.329559 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.329638 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.329817 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:13.329937 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:13.330037 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.330056 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.330078 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:13:13.332150 17935 raft_consensus.cc:1261] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 1 FOLLOWER]: Refusing update from remote peer f2c9e7ffa088464db3ce4c150cfe71e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:13.332150 18203 raft_consensus.cc:1261] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 1 FOLLOWER]: Refusing update from remote peer f2c9e7ffa088464db3ce4c150cfe71e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:13.332540 18270 consensus_queue.cc:1035] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6682c5cea854ad7bb4774acbd4efacf" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 45093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:13.332697 18276 consensus_queue.cc:1035] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b78c8f3e8441e69c65fa20577c4fe7" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:13.335831 18280 mvcc.cc:205] Tried to move back new op lower bound from 7298544407885578240 to 7298544407771017216. Current Snapshot: MvccSnapshot[applied={T|T < 7298544407885578240}]
I20260619 12:13:13.336413 18281 mvcc.cc:205] Tried to move back new op lower bound from 7298544407885578240 to 7298544407771017216. Current Snapshot: MvccSnapshot[applied={T|T < 7298544407885578240 or (T in {7298544407885578240})}]
I20260619 12:13:13.337397 17790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:13.337507 17790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:13.338260 17790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1851b4649cd8475b8252abc50745d01f: Sending DeleteTablet for 3 replicas of tablet 93c8fa2bde594a79ada7558ee0b400ce
I20260619 12:13:13.338608 17915 tablet_service.cc:1514] Processing DeleteTablet for tablet 93c8fa2bde594a79ada7558ee0b400ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:13 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:46838
I20260619 12:13:13.338657 18183 tablet_service.cc:1514] Processing DeleteTablet for tablet 93c8fa2bde594a79ada7558ee0b400ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:13 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:41850
I20260619 12:13:13.338845 18049 tablet_service.cc:1514] Processing DeleteTablet for tablet 93c8fa2bde594a79ada7558ee0b400ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:13 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:36076
I20260619 12:13:13.338892 18289 tablet_replica.cc:330] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7: stopping tablet replica
I20260619 12:13:13.338991 18289 raft_consensus.cc:2229] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:13.339049 18291 tablet_replica.cc:330] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: stopping tablet replica
I20260619 12:13:13.339092 18289 raft_consensus.cc:2258] T 93c8fa2bde594a79ada7558ee0b400ce P d8b78c8f3e8441e69c65fa20577c4fe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:13.339135 18291 raft_consensus.cc:2229] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:13:13.339253 18291 raft_consensus.cc:2258] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:13.339267 18290 tablet_replica.cc:330] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: stopping tablet replica
I20260619 12:13:13.339442 18290 raft_consensus.cc:2229] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:13.339546 18290 raft_consensus.cc:2258] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:13.339628 18291 ts_tablet_manager.cc:1905] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:13.339704 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17852
I20260619 12:13:13.339941 18290 ts_tablet_manager.cc:1905] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:13.341805 18291 ts_tablet_manager.cc:1918] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:13.341838 18290 ts_tablet_manager.cc:1918] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:13.341871 18291 log.cc:1196] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/93c8fa2bde594a79ada7558ee0b400ce
I20260619 12:13:13.341905 18290 log.cc:1196] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/93c8fa2bde594a79ada7558ee0b400ce
I20260619 12:13:13.342172 18291 ts_tablet_manager.cc:1939] T 93c8fa2bde594a79ada7558ee0b400ce P f2c9e7ffa088464db3ce4c150cfe71e6: Deleting consensus metadata
I20260619 12:13:13.342185 18290 ts_tablet_manager.cc:1939] T 93c8fa2bde594a79ada7558ee0b400ce P f6682c5cea854ad7bb4774acbd4efacf: Deleting consensus metadata
I20260619 12:13:13.342595 17777 catalog_manager.cc:4841] TS f6682c5cea854ad7bb4774acbd4efacf (127.0.201.3:45093): tablet 93c8fa2bde594a79ada7558ee0b400ce (table test-table [id=b622c2a6996b48c589324dccac4df78f]) successfully deleted
I20260619 12:13:13.342657 17775 catalog_manager.cc:4841] TS f2c9e7ffa088464db3ce4c150cfe71e6 (127.0.201.2:44595): tablet 93c8fa2bde594a79ada7558ee0b400ce (table test-table [id=b622c2a6996b48c589324dccac4df78f]) successfully deleted
W20260619 12:13:13.346710 17777 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:45791 (error 108)
I20260619 12:13:13.346911 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17986
I20260619 12:13:13.347862 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.347690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:45791 (local address 127.0.0.1:46854)
0619 12:13:13.347766 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.347809 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:45791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:13:13.347985 17777 catalog_manager.cc:4584] TS d8b78c8f3e8441e69c65fa20577c4fe7 (127.0.201.1:45791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93c8fa2bde594a79ada7558ee0b400ce: Network error: Client connection negotiation failed: client connection to 127.0.201.1:45791: connect: Connection refused (error 111)
I20260619 12:13:13.351902 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18120
I20260619 12:13:13.356416 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 17758
2026-06-19T12:13:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T12:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:13Z Disabled control of system clock
I20260619 12:13:13.384213 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38293
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:38293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:13.465435 18296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:13.465663 18296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:13.465683 18296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:13.466794 18296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:13.466873 18296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:13.466888 18296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:13.466902 18296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:13.466913 18296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:13.468341 18296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:38293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:38293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:13.468561 18296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:13.468804 18296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:13.473745 18301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.473816 18302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.473748 18304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:13.473965 18296 server_base.cc:1034] running on GCE node
I20260619 12:13:13.474321 18296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:13.474640 18296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:13.475781 18296 hybrid_clock.cc:648] HybridClock initialized: now 1781871193475763 us; error 38 us; skew 500 ppm
I20260619 12:13:13.477129 18296 webserver.cc:460] Webserver started at http://127.0.201.62:42375/ using document root <none> and password file <none>
I20260619 12:13:13.477397 18296 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:13.477449 18296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:13.477602 18296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:13.478708 18296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "596e026b58f742048463ac31a243adf5"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "6f6829e416b451c5a0e2136382144a7b"
server_key_iv: "c716130eef1955a1d4a8d8e34dcbaf9b"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.479071 18296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "596e026b58f742048463ac31a243adf5"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "6f6829e416b451c5a0e2136382144a7b"
server_key_iv: "c716130eef1955a1d4a8d8e34dcbaf9b"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.480437 18296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:13.481246 18310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.481501 18296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:13.481601 18296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "596e026b58f742048463ac31a243adf5"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "6f6829e416b451c5a0e2136382144a7b"
server_key_iv: "c716130eef1955a1d4a8d8e34dcbaf9b"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.481679 18296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:13.498649 18296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:13.499003 18296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:13.499161 18296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:13.503430 18296 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:38293
I20260619 12:13:13.503465 18362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:38293 every 8 connection(s)
I20260619 12:13:13.503823 18296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:13.504798 18363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.507138 18363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5: Bootstrap starting.
I20260619 12:13:13.507772 18363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.508085 18363 log.cc:824] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.508657 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18296
I20260619 12:13:13.508764 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:13.508818 18363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5: No bootstrap required, opened a new log
I20260619 12:13:13.509006 804 external_mini_cluster.cc:1428] Setting key 454203ce3c9e7bef8ac83949a83e6051
I20260619 12:13:13.511842 18363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596e026b58f742048463ac31a243adf5" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38293 } }
I20260619 12:13:13.512053 18363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.512086 18363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596e026b58f742048463ac31a243adf5, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.512241 18363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596e026b58f742048463ac31a243adf5" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38293 } }
I20260619 12:13:13.512332 18363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:13.512373 18363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:13.512423 18363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:13.513091 18363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596e026b58f742048463ac31a243adf5" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38293 } }
I20260619 12:13:13.513221 18363 leader_election.cc:302] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 596e026b58f742048463ac31a243adf5; no voters:
I20260619 12:13:13.513391 18363 leader_election.cc:288] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:13.513469 18368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:13.513619 18368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [term 1 LEADER]: Becoming Leader. State: Replica: 596e026b58f742048463ac31a243adf5, State: Running, Role: LEADER
I20260619 12:13:13.513648 18363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:13.513708 18368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596e026b58f742048463ac31a243adf5" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38293 } }
I20260619 12:13:13.514056 18369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "596e026b58f742048463ac31a243adf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596e026b58f742048463ac31a243adf5" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38293 } } }
I20260619 12:13:13.514155 18369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:13.514434 18370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 596e026b58f742048463ac31a243adf5. Latest consensus state: current_term: 1 leader_uuid: "596e026b58f742048463ac31a243adf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596e026b58f742048463ac31a243adf5" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 38293 } } }
I20260619 12:13:13.514566 18370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:13.514743 18375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:13.515313 18375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:13.515775 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.509561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.0.201.62:38293)
0619 12:13:13.509791 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.509802 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.509835 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:13:13.510319 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.510329 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.510418 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.510579 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.510762 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.512250 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.513211 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.513856 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.514377 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.514784 (+ 407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.514790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.514790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:13.515023 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.515067 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:13.515081 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.515439 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.515473 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.515557 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260619 12:13:13.517279 18375 catalog_manager.cc:1347] Generated new cluster ID: 4431eb2c02194488a45825f6ddf64ff0
I20260619 12:13:13.517330 18375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:13.537055 18375 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:13.537843 18375 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:13.540333 18375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5: Generated new TSK 0
I20260619 12:13:13.540530 18375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:13.545128 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38293
--builtin_ntp_servers=127.0.201.20:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:13.628189 18387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:13.628448 18387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:13.628485 18387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:13.629715 18387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:13.629769 18387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:13.629813 18387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:13:13.631175 18387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:13.631503 18387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:13.631799 18387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:13.637724 18392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.637724 18393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.637732 18395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:13.638054 18387 server_base.cc:1034] running on GCE node
I20260619 12:13:13.638266 18387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:13.638518 18387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:13.639674 18387 hybrid_clock.cc:648] HybridClock initialized: now 1781871193639659 us; error 31 us; skew 500 ppm
I20260619 12:13:13.640983 18387 webserver.cc:460] Webserver started at http://127.0.201.1:36219/ using document root <none> and password file <none>
I20260619 12:13:13.641225 18387 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:13.641294 18387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:13.641445 18387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:13.642508 18387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "732722ba4a084f31b7eea9f964a01827"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "2fcbb48004fe8d4012068faddc107445"
server_key_iv: "fe79abd9b48191e5a92481c6752ec07c"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.642870 18387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "732722ba4a084f31b7eea9f964a01827"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "2fcbb48004fe8d4012068faddc107445"
server_key_iv: "fe79abd9b48191e5a92481c6752ec07c"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.644237 18387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:13.645041 18401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.645270 18387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:13.645370 18387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "732722ba4a084f31b7eea9f964a01827"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "2fcbb48004fe8d4012068faddc107445"
server_key_iv: "fe79abd9b48191e5a92481c6752ec07c"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.645458 18387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:13.671559 18387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:13.671885 18387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:13.672045 18387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:13.672293 18387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:13.672632 18387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:13.672686 18387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.672727 18387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:13.672755 18387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.678697 18387 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:46857
I20260619 12:13:13.678743 18514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:46857 every 8 connection(s)
I20260619 12:13:13.679086 18387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:13:13.680223 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18387
I20260619 12:13:13.680336 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:13:13.680536 804 external_mini_cluster.cc:1428] Setting key 05e19eaa2ed4a76a382ca587f63a5e6f
I20260619 12:13:13.681501 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38293
--builtin_ntp_servers=127.0.201.20:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:13:13.685168 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.679836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:49621 (local address 127.0.201.62:38293)
0619 12:13:13.679884 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.679887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.680080 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:13:13.680755 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.680757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.680774 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.680830 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.681583 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.682139 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.683650 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.683840 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.683900 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.684884 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.684886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.684886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:13.684892 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.684906 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:13.684910 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.684961 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.685079 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.685110 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:13:13.685477 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.679796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38293 (local address 127.0.201.1:49621)
0619 12:13:13.679961 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.679996 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.680639 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.680901 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.680905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.680914 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:13.681481 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.681486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.682254 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.682255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.683560 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.683563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.683950 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.683952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.684115 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.684693 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:13.684694 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:13.684736 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:13.684766 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:13.684983 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:13.684990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:13.684993 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.685072 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.685189 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260619 12:13:13.686123 18515 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38293
I20260619 12:13:13.686232 18515 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:13.686412 18515 heartbeater.cc:511] Master 127.0.201.62:38293 requested a full tablet report, sending...
I20260619 12:13:13.686985 18327 ts_manager.cc:194] Registered new tserver with Master: 732722ba4a084f31b7eea9f964a01827 (127.0.201.1:46857)
I20260619 12:13:13.688014 18327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.1:49621
W20260619 12:13:13.765563 18518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:13.765781 18518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:13.765805 18518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:13.766896 18518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:13.766932 18518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:13.766956 18518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:13:13.768332 18518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:13.768534 18518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:13.768853 18518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:13.774215 18523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:13.774238 18518 server_base.cc:1034] running on GCE node
W20260619 12:13:13.774178 18526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.774189 18524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:13.774564 18518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:13.774863 18518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:13.776036 18518 hybrid_clock.cc:648] HybridClock initialized: now 1781871193775995 us; error 55 us; skew 500 ppm
I20260619 12:13:13.777494 18518 webserver.cc:460] Webserver started at http://127.0.201.2:39317/ using document root <none> and password file <none>
I20260619 12:13:13.777755 18518 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:13.777801 18518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:13.777894 18518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:13.778921 18518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "6c76055b30a342f698456def1f18999e"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "298768adbcdf1b6426f83ec7d4638c37"
server_key_iv: "d90de57d77ee816632a0e50fbc4ec5b4"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.779279 18518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "6c76055b30a342f698456def1f18999e"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "298768adbcdf1b6426f83ec7d4638c37"
server_key_iv: "d90de57d77ee816632a0e50fbc4ec5b4"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.780519 18518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:13.781215 18532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.781399 18518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:13.781466 18518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "6c76055b30a342f698456def1f18999e"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "298768adbcdf1b6426f83ec7d4638c37"
server_key_iv: "d90de57d77ee816632a0e50fbc4ec5b4"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.781517 18518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:13.807734 18518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:13.808032 18518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:13.808135 18518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:13.808341 18518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:13.808692 18518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:13.808727 18518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.808751 18518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:13.808796 18518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.814527 18518 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:33327
I20260619 12:13:13.814635 18645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:33327 every 8 connection(s)
I20260619 12:13:13.814970 18518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:13:13.816864 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18518
I20260619 12:13:13.816946 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:13:13.817159 804 external_mini_cluster.cc:1428] Setting key 03ad428796f5314e0cd214edfe49a61d
I20260619 12:13:13.818166 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38293
--builtin_ntp_servers=127.0.201.20:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:13:13.821110 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.815812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:41763 (local address 127.0.201.62:38293)
0619 12:13:13.815860 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.815862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.816159 (+ 297us) server_negotiation.cc:407] Connection header received
0619 12:13:13.816710 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.816711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.816722 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.816785 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.817695 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.818261 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.819342 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.819675 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.819717 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.820793 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.820795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.820795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:13.820799 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.820845 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:13.820850 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.820910 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.821023 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.821045 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:13:13.821382 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.815826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38293 (local address 127.0.201.2:41763)
0619 12:13:13.816025 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.816059 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.816599 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.816873 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.816876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.816884 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:13.817601 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.817608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.818385 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.818385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.819232 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.819235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.819812 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.819813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.820009 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.820610 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:13.820611 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:13.820650 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:13.820680 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:13.820931 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:13.820936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:13.820939 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.821009 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.821120 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260619 12:13:13.821997 18646 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38293
I20260619 12:13:13.822103 18646 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:13.822345 18646 heartbeater.cc:511] Master 127.0.201.62:38293 requested a full tablet report, sending...
I20260619 12:13:13.822768 18327 ts_manager.cc:194] Registered new tserver with Master: 6c76055b30a342f698456def1f18999e (127.0.201.2:33327)
I20260619 12:13:13.823179 18327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.2:41763
W20260619 12:13:13.899552 18649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:13.899845 18649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:13.899883 18649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:13.901043 18649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:13.901111 18649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:13.901153 18649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:13:13.902506 18649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:13.902782 18649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:13.903115 18649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:13.908133 18654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.908141 18655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:13.908133 18657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:13.908464 18649 server_base.cc:1034] running on GCE node
I20260619 12:13:13.908668 18649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:13.908864 18649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:13.910007 18649 hybrid_clock.cc:648] HybridClock initialized: now 1781871193909994 us; error 29 us; skew 500 ppm
I20260619 12:13:13.911325 18649 webserver.cc:460] Webserver started at http://127.0.201.3:37391/ using document root <none> and password file <none>
I20260619 12:13:13.911588 18649 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:13.911656 18649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:13.911813 18649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:13.912922 18649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "d8a798737cc64c18a610102c00237ab8"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "267bd7496a93aaf7f7f532f194b0c29e"
server_key_iv: "17e8f31e4a8d321ebdb4d73b949fe0d4"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.913304 18649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "d8a798737cc64c18a610102c00237ab8"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "267bd7496a93aaf7f7f532f194b0c29e"
server_key_iv: "17e8f31e4a8d321ebdb4d73b949fe0d4"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.914716 18649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:13.915535 18663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.915738 18649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:13.915827 18649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "d8a798737cc64c18a610102c00237ab8"
format_stamp: "Formatted at 2026-06-19 12:13:13 on dist-test-slave-c910"
server_key: "267bd7496a93aaf7f7f532f194b0c29e"
server_key_iv: "17e8f31e4a8d321ebdb4d73b949fe0d4"
server_key_version: "encryptionkey@0"
I20260619 12:13:13.915908 18649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:13.925258 18649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:13.925565 18649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:13.925709 18649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:13.925936 18649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:13.926275 18649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:13.926306 18649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.926328 18649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:13.926374 18649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:13.932165 18649 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:46787
I20260619 12:13:13.932204 18776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:46787 every 8 connection(s)
I20260619 12:13:13.932572 18649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:13:13.933506 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18649
I20260619 12:13:13.933627 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:13:13.933835 804 external_mini_cluster.cc:1428] Setting key 0c51fd6340b980dddddf18dbbe9ae8b4
I20260619 12:13:13.938063 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.933441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:47603 (local address 127.0.201.62:38293)
0619 12:13:13.933522 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.933524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.933735 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:13:13.934419 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.934420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.934432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.934471 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.935301 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.935774 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.936791 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.936976 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.937026 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.937845 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.937846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.937846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:13.937850 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.937865 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:13.937868 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.937910 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.937986 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.938005 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:13:13.938310 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.933408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:38293 (local address 127.0.201.3:47603)
0619 12:13:13.933612 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.933654 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.934325 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.934554 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.934558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.934567 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:13.935151 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.935157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.935863 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.935864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.936716 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.936719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.937053 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.937054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.937167 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.937705 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:13.937706 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:13.937735 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:13.937758 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:13.937923 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:13.937927 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:13.937929 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.937982 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.938061 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260619 12:13:13.938755 18777 heartbeater.cc:347] Connected to a master server at 127.0.201.62:38293
I20260619 12:13:13.938843 18777 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:13.938985 18777 heartbeater.cc:511] Master 127.0.201.62:38293 requested a full tablet report, sending...
I20260619 12:13:13.939404 18327 ts_manager.cc:194] Registered new tserver with Master: d8a798737cc64c18a610102c00237ab8 (127.0.201.3:46787)
I20260619 12:13:13.939735 18327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.3:47603
I20260619 12:13:13.945645 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:13:13.949860 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.947041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.0.201.62:38293)
0619 12:13:13.947080 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.947082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.947183 (+ 101us) server_negotiation.cc:407] Connection header received
0619 12:13:13.947288 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.947289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.947297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.947344 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:13.947494 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.947980 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.948812 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.948994 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.949071 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.949668 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:13.949670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:13.949670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:13.949674 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:13.949686 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:13.949690 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:13.949730 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.949790 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.949807 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:13:13.951545 18327 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:13:13.953110 18327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:13:13.959570 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.955395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:46787 (local address 127.0.0.1:50218)
0619 12:13:13.955625 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.955636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.955690 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.956262 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.956263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.956266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.956310 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.956312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.957564 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.957565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.958494 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.958496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.959323 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.959324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.959384 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.959395 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.959459 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.959493 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":33,"thread_start_us":87,"threads_started":1}
I20260619 12:13:13.959607 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.955548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.0.201.3:46787)
0619 12:13:13.956065 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.956070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.956080 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:13:13.956116 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.956118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.956133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.956201 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.956370 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.957471 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.958580 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.959235 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.959306 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.959387 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.959484 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.959524 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":82,"threads_started":1}
I20260619 12:13:13.960016 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.955332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:33327 (local address 127.0.0.1:34452)
0619 12:13:13.955544 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.955586 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.955647 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.955927 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.955930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.955932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.956001 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.956003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.956986 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.956987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.957923 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.957925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.959771 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.959772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.959815 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.959817 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.959902 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.959950 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260619 12:13:13.960036 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.955464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.0.201.2:33327)
0619 12:13:13.955748 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.955753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.955763 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:13:13.955804 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.955806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.955822 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.955922 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.956083 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.956900 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.958019 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.958738 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.958806 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.958899 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.959919 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.959960 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":71,"threads_started":1}
I20260619 12:13:13.960451 18711 tablet_service.cc:1467] Processing CreateTablet for tablet 7861064bcef947f3a67869ec8b682127 (DEFAULT_TABLE table=test-table [id=61751ec7aac544968020257bbb8abb9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:13.960682 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.955553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:46857 (local address 127.0.0.1:58092)
0619 12:13:13.955973 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:13.955982 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:13.956038 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:13.956320 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:13.956321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:13.956323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:13.956368 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.956369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.957656 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.957657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.958722 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:13.958725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.960442 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.960444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:13.960506 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.960511 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:13:13.960573 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 12:13:13.960613 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":78,"threads_started":1}
I20260619 12:13:13.960726 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:13.955761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.0.201.1:46857)
0619 12:13:13.955933 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:13.955937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:13.956144 (+ 207us) server_negotiation.cc:407] Connection header received
0619 12:13:13.956189 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:13.956192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:13.956209 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:13.956441 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:13.956498 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.957569 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.958814 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:13.959576 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:13.959652 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:13.959731 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:13.960588 (+ 857us) server_negotiation.cc:299] Negotiation successful
0619 12:13:13.960620 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260619 12:13:13.960852 18711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7861064bcef947f3a67869ec8b682127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.961390 18449 tablet_service.cc:1467] Processing CreateTablet for tablet 7861064bcef947f3a67869ec8b682127 (DEFAULT_TABLE table=test-table [id=61751ec7aac544968020257bbb8abb9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:13.961688 18449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7861064bcef947f3a67869ec8b682127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.963490 18791 tablet_bootstrap.cc:492] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Bootstrap starting.
I20260619 12:13:13.963719 18580 tablet_service.cc:1467] Processing CreateTablet for tablet 7861064bcef947f3a67869ec8b682127 (DEFAULT_TABLE table=test-table [id=61751ec7aac544968020257bbb8abb9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:13.964059 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7861064bcef947f3a67869ec8b682127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:13.964139 18791 tablet_bootstrap.cc:654] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.964349 18792 tablet_bootstrap.cc:492] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827: Bootstrap starting.
I20260619 12:13:13.964583 18791 log.cc:824] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.965018 18792 tablet_bootstrap.cc:654] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.965282 18792 log.cc:824] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.965323 18791 tablet_bootstrap.cc:492] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: No bootstrap required, opened a new log
I20260619 12:13:13.965380 18791 ts_tablet_manager.cc:1397] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:13.965905 18792 tablet_bootstrap.cc:492] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827: No bootstrap required, opened a new log
I20260619 12:13:13.965984 18792 ts_tablet_manager.cc:1397] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:13.967415 18795 tablet_bootstrap.cc:492] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Bootstrap starting.
I20260619 12:13:13.967869 18791 raft_consensus.cc:348] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:13.968025 18791 raft_consensus.cc:374] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.968067 18791 raft_consensus.cc:729] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a798737cc64c18a610102c00237ab8, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.968076 18795 tablet_bootstrap.cc:654] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:13.968211 18791 consensus_queue.cc:260] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:13.968364 18795 log.cc:824] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:13.968490 18791 ts_tablet_manager.cc:1428] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:13.968464 18792 raft_consensus.cc:348] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:13.968617 18777 heartbeater.cc:503] Master 127.0.201.62:38293 was elected leader, sending a full tablet report...
I20260619 12:13:13.968629 18792 raft_consensus.cc:374] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.968670 18792 raft_consensus.cc:729] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732722ba4a084f31b7eea9f964a01827, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.968781 18792 consensus_queue.cc:260] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:13.969031 18792 ts_tablet_manager.cc:1428] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:13.969087 18515 heartbeater.cc:503] Master 127.0.201.62:38293 was elected leader, sending a full tablet report...
I20260619 12:13:13.969604 18795 tablet_bootstrap.cc:492] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: No bootstrap required, opened a new log
I20260619 12:13:13.969671 18795 ts_tablet_manager.cc:1397] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:13.971810 18795 raft_consensus.cc:348] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:13.971951 18795 raft_consensus.cc:374] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:13.971995 18795 raft_consensus.cc:729] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c76055b30a342f698456def1f18999e, State: Initialized, Role: FOLLOWER
I20260619 12:13:13.972101 18795 consensus_queue.cc:260] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:13.972338 18795 ts_tablet_manager.cc:1428] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:13.972420 18646 heartbeater.cc:503] Master 127.0.201.62:38293 was elected leader, sending a full tablet report...
I20260619 12:13:14.011632 18799 raft_consensus.cc:482] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:14.011799 18799 raft_consensus.cc:504] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:14.012148 18799 leader_election.cc:288] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 732722ba4a084f31b7eea9f964a01827 (127.0.201.1:46857), d8a798737cc64c18a610102c00237ab8 (127.0.201.3:46787)
I20260619 12:13:14.015283 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:14.012392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:33525 (local address 127.0.201.1:46857)
0619 12:13:14.012438 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:14.012440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:14.012451 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:13:14.012539 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:14.012541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:14.012559 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:14.012626 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:14.012808 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.013294 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.014468 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.014823 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.014957 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:14.014984 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:14.015142 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 12:13:14.015186 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:13:14.015277 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:14.012314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:46857 (local address 127.0.201.2:33525)
0619 12:13:14.012376 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:14.012388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:14.012449 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:14.012643 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:14.012645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:14.012646 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:14.012723 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:14.012725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.013390 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.013391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:14.014365 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:14.014368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.014977 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.014978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:14.015051 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:14.015055 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:14.015121 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 12:13:14.015163 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:13:14.015440 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:14.012393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:53805 (local address 127.0.201.3:46787)
0619 12:13:14.012442 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:14.012444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:14.012640 (+ 196us) server_negotiation.cc:407] Connection header received
0619 12:13:14.012752 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:14.012753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:14.012762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:14.012819 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:14.012991 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.013518 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.014612 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.015073 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.015138 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:14.015147 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:14.015332 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 12:13:14.015379 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:13:14.015635 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:14.012335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:46787 (local address 127.0.201.2:53805)
0619 12:13:14.012579 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:14.012589 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:14.012669 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:14.012848 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:14.012849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:14.012851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:14.012920 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:14.012921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.013610 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.013611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:14.014528 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:14.014531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.015149 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.015150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:14.015250 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:14.015254 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:14.015313 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 12:13:14.015562 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260619 12:13:14.015745 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7861064bcef947f3a67869ec8b682127" candidate_uuid: "6c76055b30a342f698456def1f18999e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732722ba4a084f31b7eea9f964a01827" is_pre_election: true
I20260619 12:13:14.015949 18469 raft_consensus.cc:2454] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c76055b30a342f698456def1f18999e in term 0.
I20260619 12:13:14.016186 18534 leader_election.cc:302] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c76055b30a342f698456def1f18999e, 732722ba4a084f31b7eea9f964a01827; no voters:
I20260619 12:13:14.016192 18731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7861064bcef947f3a67869ec8b682127" candidate_uuid: "6c76055b30a342f698456def1f18999e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a798737cc64c18a610102c00237ab8" is_pre_election: true
I20260619 12:13:14.016328 18731 raft_consensus.cc:2454] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c76055b30a342f698456def1f18999e in term 0.
I20260619 12:13:14.016390 18799 raft_consensus.cc:2781] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:13:14.016476 18799 raft_consensus.cc:482] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:13:14.016506 18799 raft_consensus.cc:3037] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:14.017328 18799 raft_consensus.cc:504] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:14.017485 18799 leader_election.cc:288] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [CANDIDATE]: Term 1 election: Requested vote from peers 732722ba4a084f31b7eea9f964a01827 (127.0.201.1:46857), d8a798737cc64c18a610102c00237ab8 (127.0.201.3:46787)
I20260619 12:13:14.017652 18731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7861064bcef947f3a67869ec8b682127" candidate_uuid: "6c76055b30a342f698456def1f18999e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a798737cc64c18a610102c00237ab8"
I20260619 12:13:14.017642 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7861064bcef947f3a67869ec8b682127" candidate_uuid: "6c76055b30a342f698456def1f18999e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732722ba4a084f31b7eea9f964a01827"
I20260619 12:13:14.017727 18731 raft_consensus.cc:3037] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:14.017774 18469 raft_consensus.cc:3037] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:14.018711 18731 raft_consensus.cc:2454] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c76055b30a342f698456def1f18999e in term 1.
I20260619 12:13:14.018711 18469 raft_consensus.cc:2454] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c76055b30a342f698456def1f18999e in term 1.
I20260619 12:13:14.019024 18535 leader_election.cc:302] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c76055b30a342f698456def1f18999e, d8a798737cc64c18a610102c00237ab8; no voters:
I20260619 12:13:14.019127 18799 raft_consensus.cc:2781] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:14.019367 18799 raft_consensus.cc:686] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 1 LEADER]: Becoming Leader. State: Replica: 6c76055b30a342f698456def1f18999e, State: Running, Role: LEADER
I20260619 12:13:14.019495 18799 consensus_queue.cc:237] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } }
I20260619 12:13:14.020298 18327 catalog_manager.cc:5495] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c76055b30a342f698456def1f18999e (127.0.201.2). New cstate: current_term: 1 leader_uuid: "6c76055b30a342f698456def1f18999e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c76055b30a342f698456def1f18999e" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 33327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 } health_report { overall_health: UNKNOWN } } }
I20260619 12:13:14.031747 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:14.029502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.0.201.2:33327)
0619 12:13:14.029558 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:14.029560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:14.029569 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:14.029607 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:14.029608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:14.029624 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:14.029680 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:14.029793 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.030238 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.031046 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:14.031291 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:14.031356 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:14.031478 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:14.031570 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:14.031629 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:14.031661 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 12:13:14.031677 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:13:14.033470 18469 raft_consensus.cc:1261] T 7861064bcef947f3a67869ec8b682127 P 732722ba4a084f31b7eea9f964a01827 [term 1 FOLLOWER]: Refusing update from remote peer 6c76055b30a342f698456def1f18999e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:14.033519 18731 raft_consensus.cc:1261] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 1 FOLLOWER]: Refusing update from remote peer 6c76055b30a342f698456def1f18999e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:14.033816 18799 consensus_queue.cc:1035] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8a798737cc64c18a610102c00237ab8" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 46787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:14.033977 18801 consensus_queue.cc:1035] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [LEADER]: Connected to new peer: Peer: permanent_uuid: "732722ba4a084f31b7eea9f964a01827" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:14.036409 18804 mvcc.cc:205] Tried to move back new op lower bound from 7298544410758336512 to 7298544410704371712. Current Snapshot: MvccSnapshot[applied={T|T < 7298544410758336512}]
I20260619 12:13:14.038955 18326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:14.039060 18326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:14.039832 18326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 596e026b58f742048463ac31a243adf5: Sending DeleteTablet for 3 replicas of tablet 7861064bcef947f3a67869ec8b682127
I20260619 12:13:14.040187 18580 tablet_service.cc:1514] Processing DeleteTablet for tablet 7861064bcef947f3a67869ec8b682127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:14 UTC) from {username='slave'} at 127.0.0.1:34452
I20260619 12:13:14.040191 18449 tablet_service.cc:1514] Processing DeleteTablet for tablet 7861064bcef947f3a67869ec8b682127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:14 UTC) from {username='slave'} at 127.0.0.1:58092
I20260619 12:13:14.040252 18711 tablet_service.cc:1514] Processing DeleteTablet for tablet 7861064bcef947f3a67869ec8b682127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:14 UTC) from {username='slave'} at 127.0.0.1:50218
I20260619 12:13:14.040426 18815 tablet_replica.cc:330] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: stopping tablet replica
I20260619 12:13:14.040522 18815 raft_consensus.cc:2229] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:13:14.040617 18815 raft_consensus.cc:2258] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:14.040696 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18387
I20260619 12:13:14.041049 18815 ts_tablet_manager.cc:1905] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:14.043068 18815 ts_tablet_manager.cc:1918] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:14.043145 18815 log.cc:1196] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/7861064bcef947f3a67869ec8b682127
I20260619 12:13:14.043434 18815 ts_tablet_manager.cc:1939] T 7861064bcef947f3a67869ec8b682127 P 6c76055b30a342f698456def1f18999e: Deleting consensus metadata
I20260619 12:13:14.043823 18817 tablet_replica.cc:330] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: stopping tablet replica
I20260619 12:13:14.043844 18312 catalog_manager.cc:4841] TS 6c76055b30a342f698456def1f18999e (127.0.201.2:33327): tablet 7861064bcef947f3a67869ec8b682127 (table test-table [id=61751ec7aac544968020257bbb8abb9c]) successfully deleted
I20260619 12:13:14.043929 18817 raft_consensus.cc:2229] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:14.044037 18817 raft_consensus.cc:2258] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:14.044431 18817 ts_tablet_manager.cc:1905] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:14.046272 18817 ts_tablet_manager.cc:1918] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:14.046347 18817 log.cc:1196] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/7861064bcef947f3a67869ec8b682127
I20260619 12:13:14.046598 18817 ts_tablet_manager.cc:1939] T 7861064bcef947f3a67869ec8b682127 P d8a798737cc64c18a610102c00237ab8: Deleting consensus metadata
I20260619 12:13:14.047094 18313 catalog_manager.cc:4841] TS d8a798737cc64c18a610102c00237ab8 (127.0.201.3:46787): tablet 7861064bcef947f3a67869ec8b682127 (table test-table [id=61751ec7aac544968020257bbb8abb9c]) successfully deleted
W20260619 12:13:14.048610 18312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:46857 (error 108)
I20260619 12:13:14.048811 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18518
I20260619 12:13:14.049715 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:14.049570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:46857 (local address 127.0.0.1:58104)
0619 12:13:14.049630 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:14.049675 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:46857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:13:14.049856 18312 catalog_manager.cc:4584] TS 732722ba4a084f31b7eea9f964a01827 (127.0.201.1:46857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7861064bcef947f3a67869ec8b682127: Network error: Client connection negotiation failed: client connection to 127.0.201.1:46857: connect: Connection refused (error 111)
I20260619 12:13:14.053612 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18649
I20260619 12:13:14.058182 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18296
2026-06-19T12:13:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (700 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:14 dist-test-slave-c910 krb5kdc[18821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:14 dist-test-slave-c910 krb5kdc[18821](info): set up 2 sockets
Jun 19 12:13:14 dist-test-slave-c910 krb5kdc[18821](info): commencing operation
krb5kdc: starting...
W20260619 12:13:15.098999 804 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:15.218757 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36781
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:36781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:15.299400 18834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:15.299592 18834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:15.299615 18834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:15.300699 18834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:15.300755 18834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:15.300769 18834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:15.300782 18834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:15.300794 18834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:15.302172 18834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:36781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:15.302362 18834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:15.302520 18834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:15.307415 18839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:15.307446 18840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:15.307449 18834 server_base.cc:1034] running on GCE node
W20260619 12:13:15.307431 18842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:15.307809 18834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:15.308049 18834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:15.309221 18834 hybrid_clock.cc:648] HybridClock initialized: now 1781871195309217 us; error 34 us; skew 500 ppm
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:15.311631 18834 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:15.312156 18834 webserver.cc:460] Webserver started at http://127.0.201.62:43353/ using document root <none> and password file <none>
I20260619 12:13:15.312397 18834 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:15.312487 18834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:15.312637 18834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:15.313691 18834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "de01664d41a2472eac8d3683a443f514"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "961a4dd8312debdc38e3196d26bb76ee"
server_key_iv: "7fb20d6e1f5903b83892cc9ff12becdc"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.314052 18834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "de01664d41a2472eac8d3683a443f514"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "961a4dd8312debdc38e3196d26bb76ee"
server_key_iv: "7fb20d6e1f5903b83892cc9ff12becdc"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.315425 18834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:15.316337 18849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.316529 18834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:15.316627 18834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "de01664d41a2472eac8d3683a443f514"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "961a4dd8312debdc38e3196d26bb76ee"
server_key_iv: "7fb20d6e1f5903b83892cc9ff12becdc"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.316713 18834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:15.341946 18834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:15.343950 18834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:15.344113 18834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:15.348353 18834 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:36781
I20260619 12:13:15.348387 18901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:36781 every 8 connection(s)
I20260619 12:13:15.348740 18834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:15.349606 18902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:15.351897 18902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514: Bootstrap starting.
I20260619 12:13:15.352428 18902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:15.352676 18902 log.cc:824] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:15.353320 18902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514: No bootstrap required, opened a new log
I20260619 12:13:15.353622 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18834
I20260619 12:13:15.353734 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:15.353945 804 external_mini_cluster.cc:1428] Setting key bc3067f21b07c1f612c933470c915cc4
I20260619 12:13:15.355512 18902 raft_consensus.cc:348] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01664d41a2472eac8d3683a443f514" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36781 } }
I20260619 12:13:15.355681 18902 raft_consensus.cc:374] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:15.355719 18902 raft_consensus.cc:729] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de01664d41a2472eac8d3683a443f514, State: Initialized, Role: FOLLOWER
I20260619 12:13:15.355857 18902 consensus_queue.cc:260] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01664d41a2472eac8d3683a443f514" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36781 } }
I20260619 12:13:15.355942 18902 raft_consensus.cc:388] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:15.355978 18902 raft_consensus.cc:482] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:15.356026 18902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:15.357043 18902 raft_consensus.cc:504] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01664d41a2472eac8d3683a443f514" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36781 } }
I20260619 12:13:15.357131 18902 leader_election.cc:302] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de01664d41a2472eac8d3683a443f514; no voters:
I20260619 12:13:15.357347 18902 leader_election.cc:288] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:15.357369 18907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:15.357487 18907 raft_consensus.cc:686] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [term 1 LEADER]: Becoming Leader. State: Replica: de01664d41a2472eac8d3683a443f514, State: Running, Role: LEADER
I20260619 12:13:15.357568 18907 consensus_queue.cc:237] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01664d41a2472eac8d3683a443f514" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36781 } }
I20260619 12:13:15.357620 18902 sys_catalog.cc:564] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:15.357872 18907 sys_catalog.cc:455] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de01664d41a2472eac8d3683a443f514" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01664d41a2472eac8d3683a443f514" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36781 } } }
I20260619 12:13:15.357939 18907 sys_catalog.cc:458] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:15.358062 18907 sys_catalog.cc:455] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de01664d41a2472eac8d3683a443f514. Latest consensus state: current_term: 1 leader_uuid: "de01664d41a2472eac8d3683a443f514" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01664d41a2472eac8d3683a443f514" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36781 } } }
I20260619 12:13:15.358116 18907 sys_catalog.cc:458] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:15.358249 18912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:15.358736 18912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:15.360565 18912 catalog_manager.cc:1347] Generated new cluster ID: b21c09bfd7054baca4eae421a2946041
I20260619 12:13:15.360628 18912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:15.365916 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.354463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.0.201.62:36781)
0619 12:13:15.354652 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.354659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.354692 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:13:15.355497 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.355504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.355553 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.355667 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:15.356470 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.357544 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.358568 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.359283 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.359391 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.362236 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:15.362240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:15.362240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:15.362252 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:15.364236 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.364662 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.364665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.364665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.364797 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.365013 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.365013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.365014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.365306 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:15.365457 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.365589 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.365718 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260619 12:13:15.376109 18912 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:15.376652 18912 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:15.384018 18912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514: Generated new TSK 0
I20260619 12:13:15.384222 18912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:13:15.419401 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36781
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:15.501845 18928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:15.502058 18928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:15.502082 18928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:15.503160 18928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:15.503222 18928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:15.503250 18928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:13:15.504779 18928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:15.504973 18928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:15.505154 18928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:15.510136 18936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:15.510279 18933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:15.510136 18934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:15.510375 18928 server_base.cc:1034] running on GCE node
I20260619 12:13:15.510623 18928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:15.510953 18928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:15.512108 18928 hybrid_clock.cc:648] HybridClock initialized: now 1781871195512070 us; error 49 us; skew 500 ppm
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:15.514829 18928 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:13:15.515295 18928 webserver.cc:460] Webserver started at http://127.0.201.1:43925/ using document root <none> and password file <none>
I20260619 12:13:15.515492 18928 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:15.515537 18928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:15.515625 18928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:15.516625 18928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "9475c530ead74f09a14515378da57960"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "51d58444596ca9e67b0116e8d0c016b3"
server_key_iv: "5c32efed58141b42dea61d5e72f0849d"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.516948 18928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "9475c530ead74f09a14515378da57960"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "51d58444596ca9e67b0116e8d0c016b3"
server_key_iv: "5c32efed58141b42dea61d5e72f0849d"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.518198 18928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:15.518949 18943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.519115 18928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:15.519235 18928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "9475c530ead74f09a14515378da57960"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "51d58444596ca9e67b0116e8d0c016b3"
server_key_iv: "5c32efed58141b42dea61d5e72f0849d"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.519312 18928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:15.536477 18928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:15.538388 18928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:15.538549 18928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:15.538791 18928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:15.539119 18928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:15.539157 18928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.539179 18928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:15.539249 18928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.544940 18928 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:34235
I20260619 12:13:15.545037 19056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:34235 every 8 connection(s)
I20260619 12:13:15.545356 18928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:15.554476 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 18928
I20260619 12:13:15.554600 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:13:15.554838 804 external_mini_cluster.cc:1428] Setting key 7bffae6e734683cc512b3cc2faea3c99
I20260619 12:13:15.554948 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.546135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:45057 (local address 127.0.201.62:36781)
0619 12:13:15.546208 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.546210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.546437 (+ 227us) server_negotiation.cc:407] Connection header received
0619 12:13:15.547537 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.547539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.547556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.547605 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:15.548265 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.548773 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.549811 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.550036 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.550123 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.552314 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:15.552316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:15.552316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:15.552326 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:15.553592 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.553966 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.553967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.553967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.554018 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.554223 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.554223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.554223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.554323 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:15.554385 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.554760 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.554837 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:13:15.555240 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.546121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36781 (local address 127.0.201.1:45057)
0619 12:13:15.546308 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:15.546344 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:15.547412 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:15.547680 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:15.547683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:15.547967 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:15.548164 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.548168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.548895 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.548901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.549707 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.549709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.550133 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.550134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.550263 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.550826 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:15.550835 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:15.552219 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:15.553665 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:15.553671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:15.553672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:15.553900 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:15.554074 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:15.554075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:15.554075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:15.554171 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:15.554398 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:15.554406 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:15.554584 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 12:13:15.554743 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 12:13:15.554970 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260619 12:13:15.555853 19057 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36781
I20260619 12:13:15.555954 19057 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:15.556155 19057 heartbeater.cc:511] Master 127.0.201.62:36781 requested a full tablet report, sending...
I20260619 12:13:15.556631 18866 ts_manager.cc:194] Registered new tserver with Master: 9475c530ead74f09a14515378da57960 (127.0.201.1:34235)
I20260619 12:13:15.557598 18866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:45057
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:13:15.581004 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36781
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:15.661761 19062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:15.661974 19062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:15.661998 19062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:15.663143 19062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:15.663185 19062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:15.663228 19062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:13:15.664731 19062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:15.664987 19062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:15.665234 19062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:15.670390 19068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:15.670389 19067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:15.670471 19070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:15.670568 19062 server_base.cc:1034] running on GCE node
I20260619 12:13:15.670961 19062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:15.671262 19062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:15.672432 19062 hybrid_clock.cc:648] HybridClock initialized: now 1781871195672411 us; error 39 us; skew 500 ppm
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:15.674984 19062 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:13:15.675452 19062 webserver.cc:460] Webserver started at http://127.0.201.2:44963/ using document root <none> and password file <none>
I20260619 12:13:15.675647 19062 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:15.675688 19062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:15.675774 19062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:15.676774 19062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "f763f2235567411abd2629cd99611c56"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "de9f9057961f72a134ca63e448fb5411"
server_key_iv: "80eeff4bd71ff04020186a7b4d5d1fd1"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.677157 19062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "f763f2235567411abd2629cd99611c56"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "de9f9057961f72a134ca63e448fb5411"
server_key_iv: "80eeff4bd71ff04020186a7b4d5d1fd1"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.678432 19062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:13:15.679219 19077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.679383 19062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.679488 19062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "f763f2235567411abd2629cd99611c56"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "de9f9057961f72a134ca63e448fb5411"
server_key_iv: "80eeff4bd71ff04020186a7b4d5d1fd1"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.679571 19062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:15.693670 19062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:15.695688 19062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:15.695860 19062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:15.696085 19062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:15.696389 19062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:15.696442 19062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.696476 19062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:15.696503 19062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.703904 19062 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:36529
I20260619 12:13:15.704340 19062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:13:15.706183 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19062
I20260619 12:13:15.706327 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:13:15.706529 804 external_mini_cluster.cc:1428] Setting key f4b5ba7dbc35588b1ee049ce62d17e3b
I20260619 12:13:15.704698 19190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:36529 every 8 connection(s)
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
I20260619 12:13:15.716076 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.705074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:54145 (local address 127.0.201.62:36781)
0619 12:13:15.705122 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.705123 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.705406 (+ 283us) server_negotiation.cc:407] Connection header received
0619 12:13:15.706721 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.706722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.706736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.706787 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:15.707699 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.708587 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.710047 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.710262 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.710351 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.713111 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:15.713113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:15.713113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:15.713120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:15.714583 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.715054 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.715055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.715055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.715134 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.715423 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.715424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.715425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.715543 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:15.715621 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.715899 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.715981 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:15.716336 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.705071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36781 (local address 127.0.201.2:54145)
0619 12:13:15.705270 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:15.705319 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:15.706597 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:15.706861 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:15.706866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:15.707280 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:15.707576 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.707583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.708710 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.708712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.709951 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.709954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.710360 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.710361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.710495 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.711055 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:15.711064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:15.713013 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:15.714673 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:15.714680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:15.714683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:15.714975 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:15.715216 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:15.715217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:15.715217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:15.715342 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:15.715611 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:15.715618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:15.715764 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 12:13:15.715866 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:13:15.716037 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260619 12:13:15.716938 19191 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36781
I20260619 12:13:15.717024 19191 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:15.717212 19191 heartbeater.cc:511] Master 127.0.201.62:36781 requested a full tablet report, sending...
I20260619 12:13:15.717559 18866 ts_manager.cc:194] Registered new tserver with Master: f763f2235567411abd2629cd99611c56 (127.0.201.2:36529)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
I20260619 12:13:15.718102 18866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:54145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:13:15.734261 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36781
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:38215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:15.817121 19196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:15.817340 19196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:15.817366 19196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:15.818539 19196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:15.818589 19196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:15.818648 19196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:13:15.820360 19196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:38215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:36781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:15.820611 19196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:15.820869 19196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:15.826095 19201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:15.826082 19202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:15.826221 19196 server_base.cc:1034] running on GCE node
W20260619 12:13:15.826082 19204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:15.826560 19196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:15.826758 19196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:15.827889 19196 hybrid_clock.cc:648] HybridClock initialized: now 1781871195827866 us; error 43 us; skew 500 ppm
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:15.830437 19196 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:13:15.830852 19196 webserver.cc:460] Webserver started at http://127.0.201.3:42351/ using document root <none> and password file <none>
I20260619 12:13:15.831086 19196 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:15.831156 19196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:15.831336 19196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:15.832365 19196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "6ae5024e84a34aed9224b78230cdb4a7"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "6a4ffab6d67241bd686626f3215d4a25"
server_key_iv: "c38c25231cecab4cdc00dfb15fa94f3d"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.832751 19196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "6ae5024e84a34aed9224b78230cdb4a7"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "6a4ffab6d67241bd686626f3215d4a25"
server_key_iv: "c38c25231cecab4cdc00dfb15fa94f3d"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.834061 19196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:15.834740 19211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.834930 19196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:15.835018 19196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "6ae5024e84a34aed9224b78230cdb4a7"
format_stamp: "Formatted at 2026-06-19 12:13:15 on dist-test-slave-c910"
server_key: "6a4ffab6d67241bd686626f3215d4a25"
server_key_iv: "c38c25231cecab4cdc00dfb15fa94f3d"
server_key_version: "encryptionkey@0"
I20260619 12:13:15.835098 19196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:15.856513 19196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:15.858968 19196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:15.859122 19196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:15.859459 19196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:15.859781 19196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:15.859835 19196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.859875 19196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:15.859901 19196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:15.865438 19196 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:35877
I20260619 12:13:15.865477 19324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:35877 every 8 connection(s)
I20260619 12:13:15.865834 19196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:13:15.869565 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19196
I20260619 12:13:15.869645 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:13:15.869855 804 external_mini_cluster.cc:1428] Setting key 4065d09cfc586b97424c0cd90b77600f
Jun 19 12:13:15 dist-test-slave-c910 krb5kdc[18821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:15.875366 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.866588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:55459 (local address 127.0.201.62:36781)
0619 12:13:15.866630 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.866634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.866863 (+ 229us) server_negotiation.cc:407] Connection header received
0619 12:13:15.867881 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.867882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.867894 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.867941 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:15.868542 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.869041 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.870213 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.870516 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.870577 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.872698 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:15.872700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:15.872700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:15.872709 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:15.873955 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.874387 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.874388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.874389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.874462 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.874710 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.874710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.874711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.874828 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:15.874897 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.875176 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.875269 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:13:15.875566 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.866569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:36781 (local address 127.0.201.3:55459)
0619 12:13:15.866745 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:15.866781 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:15.867771 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:15.868007 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:15.868011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:15.868264 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:15.868447 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.868451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.869133 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.869134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.870132 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.870134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.870604 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.870605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.870724 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.871153 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:15.871161 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:15.872570 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:15.874064 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:15.874070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:15.874072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:15.874300 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:15.874532 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:15.874533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:15.874533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:15.874635 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:15.874899 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:15.874913 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:15.875043 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 12:13:15.875154 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 12:13:15.875332 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260619 12:13:15.876089 19325 heartbeater.cc:347] Connected to a master server at 127.0.201.62:36781
I20260619 12:13:15.876188 19325 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:15.876370 19325 heartbeater.cc:511] Master 127.0.201.62:36781 requested a full tablet report, sending...
I20260619 12:13:15.876787 18866 ts_manager.cc:194] Registered new tserver with Master: 6ae5024e84a34aed9224b78230cdb4a7 (127.0.201.3:35877)
I20260619 12:13:15.877184 18866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:55459
I20260619 12:13:15.881637 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:13:15.889714 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.883107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.0.201.62:36781)
0619 12:13:15.883174 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.883177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.883396 (+ 219us) server_negotiation.cc:407] Connection header received
0619 12:13:15.883472 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.883472 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.883481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.883532 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:15.884156 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.884648 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.885394 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.885576 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.885639 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.886991 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:15.886992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:15.886992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:15.886998 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:15.888461 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.888920 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.888922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.888922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.888977 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:15.889190 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:15.889191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:15.889191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:15.889325 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:15.889407 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.889530 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.889580 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:13:15.891554 18866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:13:15.893082 18866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:13:15.899626 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.895676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.0.201.3:35877)
0619 12:13:15.895866 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.895871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.895881 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:13:15.895921 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.895923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.895940 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.896024 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:15.896235 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.897353 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.898699 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.899182 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.899314 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.899382 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.899500 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.899542 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260619 12:13:15.899616 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.895568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:35877 (local address 127.0.0.1:55556)
0619 12:13:15.895784 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:15.895794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:15.895839 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:15.896050 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:15.896053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:15.896055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:15.896137 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.896139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.897476 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.897477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.898634 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.898635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.899325 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.899326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.899405 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.899420 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 12:13:15.899488 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:13:15.899542 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":24,"thread_start_us":68,"threads_started":1}
I20260619 12:13:15.899835 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.895371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:34235 (local address 127.0.0.1:47800)
0619 12:13:15.895642 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:15.895689 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:15.895833 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:15.896153 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:15.896153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:15.896155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:15.896203 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.896205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.897515 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.897522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.898416 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.898419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.899512 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.899514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.899588 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.899596 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:13:15.899651 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:13:15.899737 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260619 12:13:15.900079 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.895476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47800 (local address 127.0.201.1:34235)
0619 12:13:15.895974 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.895978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.895986 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:15.896051 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.896053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.896069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.896163 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:15.896260 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.897365 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.898513 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.899325 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.899408 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.899469 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.899947 (+ 478us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.899992 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":93,"threads_started":1}
I20260619 12:13:15.900168 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.895390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36529 (local address 127.0.0.1:49120)
0619 12:13:15.895684 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:15.895695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:15.895787 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:15.896131 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:15.896132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:15.896134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:15.896337 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.896339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.897237 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.897238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.898519 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:15.898521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.899514 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.899515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:15.899588 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.899595 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:13:15.899651 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 12:13:15.900090 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":430,"thread_start_us":86,"threads_started":1}
I20260619 12:13:15.900403 19259 tablet_service.cc:1467] Processing CreateTablet for tablet 4e8a867df00b4dec8bfe7c55664cf2f6 (DEFAULT_TABLE table=test-table [id=1ce4740e74e34a7bb6683cd312d2ee7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:15.900563 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:15.895503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.0.201.2:36529)
0619 12:13:15.895704 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:15.895709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:15.895900 (+ 191us) server_negotiation.cc:407] Connection header received
0619 12:13:15.895945 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:15.895948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:15.895965 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:15.896032 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:15.896428 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.897133 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.898617 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:15.899368 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:15.899432 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:15.899481 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:15.900394 (+ 913us) server_negotiation.cc:299] Negotiation successful
0619 12:13:15.900469 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
I20260619 12:13:15.900789 19259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e8a867df00b4dec8bfe7c55664cf2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:15.901191 19125 tablet_service.cc:1467] Processing CreateTablet for tablet 4e8a867df00b4dec8bfe7c55664cf2f6 (DEFAULT_TABLE table=test-table [id=1ce4740e74e34a7bb6683cd312d2ee7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:15.901377 18991 tablet_service.cc:1467] Processing CreateTablet for tablet 4e8a867df00b4dec8bfe7c55664cf2f6 (DEFAULT_TABLE table=test-table [id=1ce4740e74e34a7bb6683cd312d2ee7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:15.901511 19125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e8a867df00b4dec8bfe7c55664cf2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:15.901737 18991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e8a867df00b4dec8bfe7c55664cf2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:15.904498 19343 tablet_bootstrap.cc:492] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: Bootstrap starting.
I20260619 12:13:15.904686 19342 tablet_bootstrap.cc:492] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Bootstrap starting.
I20260619 12:13:15.904996 19344 tablet_bootstrap.cc:492] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Bootstrap starting.
I20260619 12:13:15.905299 19343 tablet_bootstrap.cc:654] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:15.905377 19342 tablet_bootstrap.cc:654] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:15.905872 19342 log.cc:824] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:15.905907 19344 tablet_bootstrap.cc:654] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:15.905910 19343 log.cc:824] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:15.906433 19344 log.cc:824] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:15.906672 19342 tablet_bootstrap.cc:492] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: No bootstrap required, opened a new log
I20260619 12:13:15.906749 19342 ts_tablet_manager.cc:1397] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:13:15.907409 19344 tablet_bootstrap.cc:492] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: No bootstrap required, opened a new log
I20260619 12:13:15.907483 19344 ts_tablet_manager.cc:1397] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:13:15.907639 19343 tablet_bootstrap.cc:492] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: No bootstrap required, opened a new log
I20260619 12:13:15.907704 19343 ts_tablet_manager.cc:1397] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:13:15.908464 19342 raft_consensus.cc:348] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:15.908656 19342 raft_consensus.cc:374] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:15.908699 19342 raft_consensus.cc:729] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae5024e84a34aed9224b78230cdb4a7, State: Initialized, Role: FOLLOWER
I20260619 12:13:15.908818 19342 consensus_queue.cc:260] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:15.909101 19342 ts_tablet_manager.cc:1428] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:13:15.909224 19325 heartbeater.cc:503] Master 127.0.201.62:36781 was elected leader, sending a full tablet report...
I20260619 12:13:15.909534 19344 raft_consensus.cc:348] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:15.909726 19344 raft_consensus.cc:374] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:15.909653 19343 raft_consensus.cc:348] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:15.909776 19344 raft_consensus.cc:729] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f763f2235567411abd2629cd99611c56, State: Initialized, Role: FOLLOWER
I20260619 12:13:15.909801 19343 raft_consensus.cc:374] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:15.909839 19343 raft_consensus.cc:729] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9475c530ead74f09a14515378da57960, State: Initialized, Role: FOLLOWER
I20260619 12:13:15.909914 19344 consensus_queue.cc:260] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:15.909960 19343 consensus_queue.cc:260] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:15.910205 19344 ts_tablet_manager.cc:1428] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:15.910205 19343 ts_tablet_manager.cc:1428] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 12:13:15.910210 19057 heartbeater.cc:503] Master 127.0.201.62:36781 was elected leader, sending a full tablet report...
I20260619 12:13:15.910209 19191 heartbeater.cc:503] Master 127.0.201.62:36781 was elected leader, sending a full tablet report...
W20260619 12:13:15.954960 19192 tablet.cc:2369] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:13:16.046142 19058 tablet.cc:2369] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:13:16.116475 19326 tablet.cc:2369] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:13:16.205972 19348 raft_consensus.cc:482] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:16.206104 19348 raft_consensus.cc:504] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:16.206472 19348 leader_election.cc:288] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9475c530ead74f09a14515378da57960 (127.0.201.1:34235), f763f2235567411abd2629cd99611c56 (127.0.201.2:36529)
I20260619 12:13:16.209475 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.206653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:34235 (local address 127.0.201.3:32797)
0619 12:13:16.206726 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.206740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.206794 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.206976 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.206977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.206979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:16.207059 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.207061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.207769 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.207770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.208565 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.208567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.209275 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.209276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.209342 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.209345 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.209393 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.209421 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 12:13:16.209527 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.206755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:32797 (local address 127.0.201.1:34235)
0619 12:13:16.206809 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.206812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.206823 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:13:16.206857 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.206858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.206870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.206941 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:16.207137 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.207689 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.208662 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.209196 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.209267 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.209278 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.209402 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.209440 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:13:16.209617 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.206702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:36529 (local address 127.0.201.3:45285)
0619 12:13:16.206913 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.206923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.206979 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.207136 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.207137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.207138 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:16.207223 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.207225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.207862 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.207863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.208597 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.208599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.209357 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.209358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.209446 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.209450 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.209503 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.209537 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
I20260619 12:13:16.209667 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.206823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:45285 (local address 127.0.201.2:36529)
0619 12:13:16.206882 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.206884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.206988 (+ 104us) server_negotiation.cc:407] Connection header received
0619 12:13:16.207023 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.207025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.207039 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.207101 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:16.207292 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.207779 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.208725 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.209283 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.209321 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.209330 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.209535 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.209585 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:13:16.210041 19128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8a867df00b4dec8bfe7c55664cf2f6" candidate_uuid: "6ae5024e84a34aed9224b78230cdb4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f763f2235567411abd2629cd99611c56" is_pre_election: true
I20260619 12:13:16.210084 19011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8a867df00b4dec8bfe7c55664cf2f6" candidate_uuid: "6ae5024e84a34aed9224b78230cdb4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9475c530ead74f09a14515378da57960" is_pre_election: true
I20260619 12:13:16.210220 19128 raft_consensus.cc:2454] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae5024e84a34aed9224b78230cdb4a7 in term 0.
I20260619 12:13:16.210232 19011 raft_consensus.cc:2454] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae5024e84a34aed9224b78230cdb4a7 in term 0.
I20260619 12:13:16.210459 19212 leader_election.cc:302] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae5024e84a34aed9224b78230cdb4a7, f763f2235567411abd2629cd99611c56; no voters:
I20260619 12:13:16.210608 19348 raft_consensus.cc:2781] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:13:16.210664 19348 raft_consensus.cc:482] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:13:16.210712 19348 raft_consensus.cc:3037] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:16.211509 19348 raft_consensus.cc:504] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:16.211650 19348 leader_election.cc:288] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [CANDIDATE]: Term 1 election: Requested vote from peers 9475c530ead74f09a14515378da57960 (127.0.201.1:34235), f763f2235567411abd2629cd99611c56 (127.0.201.2:36529)
I20260619 12:13:16.211851 19011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8a867df00b4dec8bfe7c55664cf2f6" candidate_uuid: "6ae5024e84a34aed9224b78230cdb4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9475c530ead74f09a14515378da57960"
I20260619 12:13:16.211870 19128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e8a867df00b4dec8bfe7c55664cf2f6" candidate_uuid: "6ae5024e84a34aed9224b78230cdb4a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f763f2235567411abd2629cd99611c56"
I20260619 12:13:16.211937 19128 raft_consensus.cc:3037] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:16.211946 19011 raft_consensus.cc:3037] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:16.212639 19011 raft_consensus.cc:2454] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae5024e84a34aed9224b78230cdb4a7 in term 1.
I20260619 12:13:16.212683 19128 raft_consensus.cc:2454] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae5024e84a34aed9224b78230cdb4a7 in term 1.
I20260619 12:13:16.212852 19214 leader_election.cc:302] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae5024e84a34aed9224b78230cdb4a7, 9475c530ead74f09a14515378da57960; no voters:
I20260619 12:13:16.212981 19348 raft_consensus.cc:2781] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:16.213148 19348 raft_consensus.cc:686] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae5024e84a34aed9224b78230cdb4a7, State: Running, Role: LEADER
I20260619 12:13:16.213274 19348 consensus_queue.cc:237] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } }
I20260619 12:13:16.214130 18864 catalog_manager.cc:5495] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ae5024e84a34aed9224b78230cdb4a7 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "6ae5024e84a34aed9224b78230cdb4a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ae5024e84a34aed9224b78230cdb4a7" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35877 } health_report { overall_health: HEALTHY } } }
I20260619 12:13:16.256469 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.253338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.0.201.3:35877)
0619 12:13:16.253523 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.253524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.253531 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:13:16.253564 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.253564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.253582 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.253667 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:16.253830 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.254315 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.255118 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.255444 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.255513 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.255919 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:16.256051 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:16.256327 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.256357 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.256369 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260619 12:13:16.261426 19128 raft_consensus.cc:1261] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 1 FOLLOWER]: Refusing update from remote peer 6ae5024e84a34aed9224b78230cdb4a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:16.261513 19011 raft_consensus.cc:1261] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 9475c530ead74f09a14515378da57960 [term 1 FOLLOWER]: Refusing update from remote peer 6ae5024e84a34aed9224b78230cdb4a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:16.262081 19352 consensus_queue.cc:1035] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9475c530ead74f09a14515378da57960" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:16.262274 19348 consensus_queue.cc:1035] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f763f2235567411abd2629cd99611c56" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:16.265180 19356 mvcc.cc:205] Tried to move back new op lower bound from 7298544419883786240 to 7298544419690143744. Current Snapshot: MvccSnapshot[applied={T|T < 7298544419883786240}]
I20260619 12:13:16.265362 19355 mvcc.cc:205] Tried to move back new op lower bound from 7298544419883786240 to 7298544419690143744. Current Snapshot: MvccSnapshot[applied={T|T < 7298544419883786240 or (T in {7298544419883786240})}]
I20260619 12:13:16.265568 19357 mvcc.cc:205] Tried to move back new op lower bound from 7298544419883786240 to 7298544419690143744. Current Snapshot: MvccSnapshot[applied={T|T < 7298544419883786240}]
I20260619 12:13:16.267323 18864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:16.267419 18864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:16.268148 18864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de01664d41a2472eac8d3683a443f514: Sending DeleteTablet for 3 replicas of tablet 4e8a867df00b4dec8bfe7c55664cf2f6
I20260619 12:13:16.268535 19125 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e8a867df00b4dec8bfe7c55664cf2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:16 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49120
I20260619 12:13:16.268582 19259 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e8a867df00b4dec8bfe7c55664cf2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:16 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:55556
I20260619 12:13:16.268755 18991 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e8a867df00b4dec8bfe7c55664cf2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:16 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:47800
I20260619 12:13:16.268786 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18928
I20260619 12:13:16.268793 19365 tablet_replica.cc:330] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: stopping tablet replica
I20260619 12:13:16.268891 19365 raft_consensus.cc:2229] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:16.268986 19365 raft_consensus.cc:2258] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:16.269630 19365 ts_tablet_manager.cc:1905] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:16.271337 19365 ts_tablet_manager.cc:1918] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:16.271407 19365 log.cc:1196] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/4e8a867df00b4dec8bfe7c55664cf2f6
I20260619 12:13:16.271674 19365 ts_tablet_manager.cc:1939] T 4e8a867df00b4dec8bfe7c55664cf2f6 P f763f2235567411abd2629cd99611c56: Deleting consensus metadata
I20260619 12:13:16.272235 18850 catalog_manager.cc:4841] TS f763f2235567411abd2629cd99611c56 (127.0.201.2:36529): tablet 4e8a867df00b4dec8bfe7c55664cf2f6 (table test-table [id=1ce4740e74e34a7bb6683cd312d2ee7c]) successfully deleted
I20260619 12:13:16.277058 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19062
W20260619 12:13:16.277884 18852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:34235 (error 108)
I20260619 12:13:16.278407 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.278262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:34235 (local address 127.0.0.1:47808)
0619 12:13:16.278318 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.278366 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:34235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 12:13:16.278596 18852 catalog_manager.cc:4584] TS 9475c530ead74f09a14515378da57960 (127.0.201.1:34235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e8a867df00b4dec8bfe7c55664cf2f6: Network error: Client connection negotiation failed: client connection to 127.0.201.1:34235: connect: Connection refused (error 111)
I20260619 12:13:16.279304 19366 tablet_replica.cc:330] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: stopping tablet replica
I20260619 12:13:16.279438 19366 raft_consensus.cc:2229] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:13:16.279554 19366 raft_consensus.cc:2258] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:16.279982 19366 ts_tablet_manager.cc:1905] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:16.281494 19366 ts_tablet_manager.cc:1918] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:16.281585 19366 log.cc:1196] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/4e8a867df00b4dec8bfe7c55664cf2f6
I20260619 12:13:16.281840 19366 ts_tablet_manager.cc:1939] T 4e8a867df00b4dec8bfe7c55664cf2f6 P 6ae5024e84a34aed9224b78230cdb4a7: Deleting consensus metadata
I20260619 12:13:16.282277 18852 catalog_manager.cc:4841] TS 6ae5024e84a34aed9224b78230cdb4a7 (127.0.201.3:35877): tablet 4e8a867df00b4dec8bfe7c55664cf2f6 (table test-table [id=1ce4740e74e34a7bb6683cd312d2ee7c]) successfully deleted
I20260619 12:13:16.284157 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19196
I20260619 12:13:16.290782 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 18834
2026-06-19T12:13:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2237 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T12:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:16Z Disabled control of system clock
I20260619 12:13:16.323451 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:39705
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:39705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:16.404508 19371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:16.404752 19371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:16.404773 19371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:16.405987 19371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:16.406039 19371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:16.406054 19371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:16.406066 19371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:16.406121 19371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:16.407615 19371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:39705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:39705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:16.407867 19371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:16.408165 19371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:16.413343 19376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:16.413277 19377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:16.413347 19379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.413678 19371 server_base.cc:1034] running on GCE node
I20260619 12:13:16.413969 19371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:16.414284 19371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:16.415490 19371 hybrid_clock.cc:648] HybridClock initialized: now 1781871196415469 us; error 52 us; skew 500 ppm
I20260619 12:13:16.421195 19371 webserver.cc:460] Webserver started at http://127.0.201.62:45373/ using document root <none> and password file <none>
I20260619 12:13:16.421475 19371 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:16.421547 19371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:16.421698 19371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:16.422833 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "5ce8b7afb5614a4b90cbea8d9596657f"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "446a6442098e4e7342773181e97fb0dc"
server_key_iv: "3fb030229850a78a9a41543d3e4c2acc"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.423243 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "5ce8b7afb5614a4b90cbea8d9596657f"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "446a6442098e4e7342773181e97fb0dc"
server_key_iv: "3fb030229850a78a9a41543d3e4c2acc"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.424554 19371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:16.425273 19385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.425458 19371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:16.425570 19371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "5ce8b7afb5614a4b90cbea8d9596657f"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "446a6442098e4e7342773181e97fb0dc"
server_key_iv: "3fb030229850a78a9a41543d3e4c2acc"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.425642 19371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:16.453004 19371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:16.453363 19371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:16.453523 19371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:16.457602 19371 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:39705
I20260619 12:13:16.457638 19437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:39705 every 8 connection(s)
I20260619 12:13:16.457998 19371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:16.458264 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19371
I20260619 12:13:16.458374 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:16.458627 804 external_mini_cluster.cc:1428] Setting key 6e404e6823a46459685d1babc3559af6
I20260619 12:13:16.459273 19438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:16.462302 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f: Bootstrap starting.
I20260619 12:13:16.462949 19438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:16.463307 19438 log.cc:824] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:16.464109 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f: No bootstrap required, opened a new log
I20260619 12:13:16.464392 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.459162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.0.201.62:39705)
0619 12:13:16.459459 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.459468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.459494 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:13:16.459929 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.459935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.459994 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.460107 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:16.460336 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.461168 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.462292 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.462826 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.462924 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.463730 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:16.463734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:16.463735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:16.463930 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:16.463962 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:16.463973 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:16.464026 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.464125 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.464222 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":166,"threads_started":1}
I20260619 12:13:16.466061 19438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39705 } }
I20260619 12:13:16.466187 19438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:16.466209 19438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce8b7afb5614a4b90cbea8d9596657f, State: Initialized, Role: FOLLOWER
I20260619 12:13:16.466342 19438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39705 } }
I20260619 12:13:16.466414 19438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:16.466436 19438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:16.466501 19438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:16.467016 19438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39705 } }
I20260619 12:13:16.467126 19438 leader_election.cc:302] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ce8b7afb5614a4b90cbea8d9596657f; no voters:
I20260619 12:13:16.467316 19438 leader_election.cc:288] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:16.467347 19443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:16.467455 19443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [term 1 LEADER]: Becoming Leader. State: Replica: 5ce8b7afb5614a4b90cbea8d9596657f, State: Running, Role: LEADER
I20260619 12:13:16.467535 19443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39705 } }
I20260619 12:13:16.467579 19438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:16.467854 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ce8b7afb5614a4b90cbea8d9596657f. Latest consensus state: current_term: 1 leader_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39705 } } }
I20260619 12:13:16.467852 19444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ce8b7afb5614a4b90cbea8d9596657f" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 39705 } } }
I20260619 12:13:16.467975 19444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:16.468127 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:16.468286 19449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:16.468780 19449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:16.470224 19449 catalog_manager.cc:1347] Generated new cluster ID: 9341afce3e854291adf06bcf874e3aa4
I20260619 12:13:16.470302 19449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:16.498143 19449 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:16.498883 19449 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:16.504997 19449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f: Generated new TSK 0
I20260619 12:13:16.505213 19449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:16.510365 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39705
--builtin_ntp_servers=127.0.201.20:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:16.592954 19462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:16.593201 19462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:16.593237 19462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:16.594406 19462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:16.594477 19462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:16.594519 19462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:13:16.595975 19462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:16.596252 19462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:16.596556 19462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:16.602147 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:16.602171 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.602180 19462 server_base.cc:1034] running on GCE node
W20260619 12:13:16.602144 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.602525 19462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:16.602726 19462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:16.603868 19462 hybrid_clock.cc:648] HybridClock initialized: now 1781871196603857 us; error 30 us; skew 500 ppm
I20260619 12:13:16.605077 19462 webserver.cc:460] Webserver started at http://127.0.201.1:39843/ using document root <none> and password file <none>
I20260619 12:13:16.605361 19462 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:16.605438 19462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:16.605594 19462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:16.606670 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "a138fc7af00c623ef414a9a3b06efe56"
server_key_iv: "387dfeccf6e97a16f79085ce3f5b8870"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.607059 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "a138fc7af00c623ef414a9a3b06efe56"
server_key_iv: "387dfeccf6e97a16f79085ce3f5b8870"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.608475 19462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:16.609220 19476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.609421 19462 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:16.609530 19462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "a138fc7af00c623ef414a9a3b06efe56"
server_key_iv: "387dfeccf6e97a16f79085ce3f5b8870"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.609613 19462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:16.617971 19462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:16.618278 19462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:16.618428 19462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:16.618646 19462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:16.618970 19462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:16.619024 19462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.619066 19462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:16.619091 19462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.625260 19462 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:43183
I20260619 12:13:16.625305 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:43183 every 8 connection(s)
I20260619 12:13:16.625648 19462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:13:16.631014 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.626475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:55785 (local address 127.0.201.62:39705)
0619 12:13:16.626554 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.626556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.626786 (+ 230us) server_negotiation.cc:407] Connection header received
0619 12:13:16.627335 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.627337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.627352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.627417 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:16.628108 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.628766 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.629742 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.629967 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.630022 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.630745 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:16.630746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:16.630746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:16.630750 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:16.630762 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:16.630766 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:16.630842 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.630921 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.630952 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:13:16.631161 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.626445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:39705 (local address 127.0.201.1:55785)
0619 12:13:16.626633 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.626676 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.627149 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.627483 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.627486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.627494 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:16.628019 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.628024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.628854 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.628854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.629672 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.629675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.630041 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.630042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.630142 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.630619 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:16.630619 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:16.630647 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:16.630669 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:16.630845 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:16.630847 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:16.630849 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.630896 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.630972 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260619 12:13:16.631734 19590 heartbeater.cc:347] Connected to a master server at 127.0.201.62:39705
I20260619 12:13:16.631850 19590 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:16.632077 19590 heartbeater.cc:511] Master 127.0.201.62:39705 requested a full tablet report, sending...
I20260619 12:13:16.632593 19402 ts_manager.cc:194] Registered new tserver with Master: 36bd4ecd3b8d4575a3926b68bae0dd4d (127.0.201.1:43183)
I20260619 12:13:16.633221 19402 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.1:55785
I20260619 12:13:16.635823 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19462
I20260619 12:13:16.635914 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:13:16.636147 804 external_mini_cluster.cc:1428] Setting key 8b12d650da264814de3e83899a44d47c
I20260619 12:13:16.637095 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39705
--builtin_ntp_servers=127.0.201.20:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:16.719653 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:16.719868 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:16.719890 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:16.720963 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:16.721014 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:16.721040 19593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:13:16.722538 19593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:16.722726 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:16.722991 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:16.727973 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:16.728034 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.728022 19593 server_base.cc:1034] running on GCE node
W20260619 12:13:16.727989 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.728353 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:16.728612 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:16.729779 19593 hybrid_clock.cc:648] HybridClock initialized: now 1781871196729769 us; error 36 us; skew 500 ppm
I20260619 12:13:16.730988 19593 webserver.cc:460] Webserver started at http://127.0.201.2:33819/ using document root <none> and password file <none>
I20260619 12:13:16.731279 19593 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:16.731348 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:16.731509 19593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:16.732589 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "32db619b177f488cb4799e2a526fde3f"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "e4fd091b30a1160baee7562eb45134af"
server_key_iv: "a39e081578e1c2a7f7bf4a22c6e11a13"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.732954 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "32db619b177f488cb4799e2a526fde3f"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "e4fd091b30a1160baee7562eb45134af"
server_key_iv: "a39e081578e1c2a7f7bf4a22c6e11a13"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.734225 19593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:16.734958 19607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.735137 19593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:16.735251 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "32db619b177f488cb4799e2a526fde3f"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "e4fd091b30a1160baee7562eb45134af"
server_key_iv: "a39e081578e1c2a7f7bf4a22c6e11a13"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.735339 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:16.748340 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:16.748641 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:16.748788 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:16.749022 19593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:16.749337 19593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:16.749390 19593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.749424 19593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:16.749454 19593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.755319 19593 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:40765
I20260619 12:13:16.755389 19720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:40765 every 8 connection(s)
I20260619 12:13:16.755710 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:13:16.760954 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.756472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:45023 (local address 127.0.201.62:39705)
0619 12:13:16.756541 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.756543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.756773 (+ 230us) server_negotiation.cc:407] Connection header received
0619 12:13:16.757325 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.757326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.757338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.757391 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:16.758056 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.758550 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.759596 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.759787 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.759858 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.760660 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:16.760661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:16.760662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:16.760667 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:16.760680 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:16.760684 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:16.760753 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.760842 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.760867 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:13:16.761174 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.756447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:39705 (local address 127.0.201.2:45023)
0619 12:13:16.756656 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.756690 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.757212 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.757471 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.757475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.757486 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:16.757966 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.757970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.758645 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.758646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.759497 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.759499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.759875 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.759876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.760014 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.760527 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:16.760528 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:16.760553 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:16.760575 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:16.760752 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:16.760758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:16.760761 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.760844 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.760941 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260619 12:13:16.761686 19721 heartbeater.cc:347] Connected to a master server at 127.0.201.62:39705
I20260619 12:13:16.761804 19721 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:16.761994 19721 heartbeater.cc:511] Master 127.0.201.62:39705 requested a full tablet report, sending...
I20260619 12:13:16.762214 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19593
I20260619 12:13:16.762321 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:13:16.762401 19402 ts_manager.cc:194] Registered new tserver with Master: 32db619b177f488cb4799e2a526fde3f (127.0.201.2:40765)
I20260619 12:13:16.762593 804 external_mini_cluster.cc:1428] Setting key ced723311a8b3c2184cd7c049e7b1e85
I20260619 12:13:16.762854 19402 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.2:45023
I20260619 12:13:16.763633 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39705
--builtin_ntp_servers=127.0.201.20:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:16.842700 19724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:16.842911 19724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:16.842932 19724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:16.844105 19724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:16.844146 19724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:16.844170 19724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:13:16.845489 19724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:39705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:16.845733 19724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:16.846028 19724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:16.851025 19730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:16.851049 19732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.851233 19724 server_base.cc:1034] running on GCE node
W20260619 12:13:16.851025 19729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:16.851478 19724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:16.851727 19724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:16.852922 19724 hybrid_clock.cc:648] HybridClock initialized: now 1781871196852904 us; error 28 us; skew 500 ppm
I20260619 12:13:16.854300 19724 webserver.cc:460] Webserver started at http://127.0.201.3:37071/ using document root <none> and password file <none>
I20260619 12:13:16.854563 19724 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:16.854637 19724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:16.854792 19724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:16.855934 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "3be0f5e64b374530a5ce57c4f2e2e697"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "a9ae83f32e5ec17a1a66ab93cab2f124"
server_key_iv: "a833bf992ad4fc54532b13b1089d55cd"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.856307 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "3be0f5e64b374530a5ce57c4f2e2e697"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "a9ae83f32e5ec17a1a66ab93cab2f124"
server_key_iv: "a833bf992ad4fc54532b13b1089d55cd"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.857551 19724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:13:16.858305 19738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.858479 19724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:16.858577 19724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "3be0f5e64b374530a5ce57c4f2e2e697"
format_stamp: "Formatted at 2026-06-19 12:13:16 on dist-test-slave-c910"
server_key: "a9ae83f32e5ec17a1a66ab93cab2f124"
server_key_iv: "a833bf992ad4fc54532b13b1089d55cd"
server_key_version: "encryptionkey@0"
I20260619 12:13:16.858661 19724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:16.880527 19724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:16.880875 19724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:16.881042 19724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:16.881306 19724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:16.881649 19724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:16.881701 19724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.881742 19724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:16.881765 19724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:16.887557 19724 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:38631
I20260619 12:13:16.887599 19851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:38631 every 8 connection(s)
I20260619 12:13:16.887926 19724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:13:16.888233 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19724
I20260619 12:13:16.888326 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:13:16.888572 804 external_mini_cluster.cc:1428] Setting key 8384a9d90474eb50304c81b9e098db0e
I20260619 12:13:16.893054 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.888672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:35087 (local address 127.0.201.62:39705)
0619 12:13:16.888733 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.888735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.888965 (+ 230us) server_negotiation.cc:407] Connection header received
0619 12:13:16.889596 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.889597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.889609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.889676 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:16.890303 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.890782 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.891767 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.891968 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.892020 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.892849 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:16.892850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:16.892850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:16.892854 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:16.892863 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:16.892866 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:16.892920 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.892982 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.892999 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:13:16.893297 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.888665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:39705 (local address 127.0.201.3:35087)
0619 12:13:16.888871 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.888907 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.889497 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.889724 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.889727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.889736 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:16.890225 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.890228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.890856 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.890856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.891695 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.891697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.892039 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.892039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.892144 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.892726 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:16.892727 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:16.892755 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:16.892780 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:16.892923 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:16.892926 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:16.892928 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.892973 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.893048 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260619 12:13:16.893783 19852 heartbeater.cc:347] Connected to a master server at 127.0.201.62:39705
I20260619 12:13:16.893858 19852 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:16.894047 19852 heartbeater.cc:511] Master 127.0.201.62:39705 requested a full tablet report, sending...
I20260619 12:13:16.894430 19402 ts_manager.cc:194] Registered new tserver with Master: 3be0f5e64b374530a5ce57c4f2e2e697 (127.0.201.3:38631)
I20260619 12:13:16.894795 19402 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.3:35087
I20260619 12:13:16.900370 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:13:16.904848 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.901727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.0.201.62:39705)
0619 12:13:16.901800 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.901803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.901922 (+ 119us) server_negotiation.cc:407] Connection header received
0619 12:13:16.901970 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.901971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.901982 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.902033 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:16.902197 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.902676 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.903504 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.903711 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.903787 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.904531 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:16.904533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:16.904533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:16.904537 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:16.904557 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:16.904560 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:16.904650 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.904751 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.904761 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:13:16.906302 19402 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:13:16.907999 19402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:13:16.914535 19863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.910261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48704 (local address 127.0.201.3:38631)
0619 12:13:16.910439 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.910444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.910605 (+ 161us) server_negotiation.cc:407] Connection header received
0619 12:13:16.910646 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.910648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.910661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.910751 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:16.911006 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.912184 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.913589 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.914119 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.914212 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.914317 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.914401 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.914442 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
I20260619 12:13:16.914561 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.910138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:43183 (local address 127.0.0.1:56514)
0619 12:13:16.910350 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.910391 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.910458 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.910933 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.910934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.910937 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:16.910993 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.910995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.912013 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.912014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.913371 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.913374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.914263 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.914263 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.914341 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.914348 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.914382 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.914455 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":25,"thread_start_us":70,"threads_started":1}
I20260619 12:13:16.914702 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.910580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.0.201.2:40765)
0619 12:13:16.910759 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.910763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.910771 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:16.910831 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.910833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.910847 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.911055 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:16.911106 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.912189 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.913359 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.914155 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.914441 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.914539 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.914564 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.914608 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260619 12:13:16.914819 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.910346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:40765 (local address 127.0.0.1:44720)
0619 12:13:16.910568 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.910577 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.910617 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.910946 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.910946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.910948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:16.910984 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.910985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.912330 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.912331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.913262 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.913265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.914261 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.914262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.914341 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.914350 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.914382 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.914751 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":319,"thread_start_us":76,"threads_started":1}
I20260619 12:13:16.915143 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.910233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.0.201.1:43183)
0619 12:13:16.910733 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:16.910737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:16.910746 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:16.910791 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:16.910793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:16.910826 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:16.911070 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:16.911135 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.911885 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.913468 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.914155 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.914939 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.915033 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:16.915053 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 12:13:16.915084 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":80,"threads_started":1}
I20260619 12:13:16.914565 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:16.910162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:38631 (local address 127.0.0.1:48704)
0619 12:13:16.910470 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:16.910481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:16.910538 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:16.910838 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:16.910842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:16.910844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:16.910925 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.910927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.912297 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.912298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.913503 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:16.913505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:16.914213 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:16.914213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:16.914298 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:16.914312 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 12:13:16.914388 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:13:16.914464 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":70,"thread_start_us":108,"threads_started":1}
I20260619 12:13:16.915706 19655 tablet_service.cc:1467] Processing CreateTablet for tablet a025c7584f8c4db5bfddc0b36f50ea76 (DEFAULT_TABLE table=test-table [id=f713a514d1e7438898bf99f60943ce5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:16.915797 19524 tablet_service.cc:1467] Processing CreateTablet for tablet a025c7584f8c4db5bfddc0b36f50ea76 (DEFAULT_TABLE table=test-table [id=f713a514d1e7438898bf99f60943ce5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:16.916119 19655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a025c7584f8c4db5bfddc0b36f50ea76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:16.916139 19524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a025c7584f8c4db5bfddc0b36f50ea76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:16.918588 19867 tablet_bootstrap.cc:492] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Bootstrap starting.
I20260619 12:13:16.919057 19786 tablet_service.cc:1467] Processing CreateTablet for tablet a025c7584f8c4db5bfddc0b36f50ea76 (DEFAULT_TABLE table=test-table [id=f713a514d1e7438898bf99f60943ce5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:16.919356 19867 tablet_bootstrap.cc:654] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:16.919358 19866 tablet_bootstrap.cc:492] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: Bootstrap starting.
I20260619 12:13:16.919395 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a025c7584f8c4db5bfddc0b36f50ea76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:16.919654 19867 log.cc:824] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:16.919974 19866 tablet_bootstrap.cc:654] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:16.920281 19866 log.cc:824] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:16.920377 19867 tablet_bootstrap.cc:492] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: No bootstrap required, opened a new log
I20260619 12:13:16.920444 19867 ts_tablet_manager.cc:1397] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:13:16.921090 19866 tablet_bootstrap.cc:492] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: No bootstrap required, opened a new log
I20260619 12:13:16.921166 19866 ts_tablet_manager.cc:1397] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:16.922746 19870 tablet_bootstrap.cc:492] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Bootstrap starting.
I20260619 12:13:16.922758 19867 raft_consensus.cc:348] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:16.922931 19867 raft_consensus.cc:374] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:16.922969 19867 raft_consensus.cc:729] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32db619b177f488cb4799e2a526fde3f, State: Initialized, Role: FOLLOWER
I20260619 12:13:16.923097 19867 consensus_queue.cc:260] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:16.923398 19867 ts_tablet_manager.cc:1428] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 12:13:16.923509 19870 tablet_bootstrap.cc:654] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:16.923521 19721 heartbeater.cc:503] Master 127.0.201.62:39705 was elected leader, sending a full tablet report...
I20260619 12:13:16.923834 19870 log.cc:824] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:16.923889 19866 raft_consensus.cc:348] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:16.924068 19866 raft_consensus.cc:374] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:16.924114 19866 raft_consensus.cc:729] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36bd4ecd3b8d4575a3926b68bae0dd4d, State: Initialized, Role: FOLLOWER
I20260619 12:13:16.924226 19866 consensus_queue.cc:260] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:16.924505 19866 ts_tablet_manager.cc:1428] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 12:13:16.924537 19590 heartbeater.cc:503] Master 127.0.201.62:39705 was elected leader, sending a full tablet report...
I20260619 12:13:16.925128 19870 tablet_bootstrap.cc:492] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: No bootstrap required, opened a new log
I20260619 12:13:16.925212 19870 ts_tablet_manager.cc:1397] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 12:13:16.926930 19870 raft_consensus.cc:348] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:16.927048 19870 raft_consensus.cc:374] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:16.927069 19870 raft_consensus.cc:729] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be0f5e64b374530a5ce57c4f2e2e697, State: Initialized, Role: FOLLOWER
I20260619 12:13:16.927175 19870 consensus_queue.cc:260] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:16.927429 19870 ts_tablet_manager.cc:1428] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:13:16.927500 19852 heartbeater.cc:503] Master 127.0.201.62:39705 was elected leader, sending a full tablet report...
W20260619 12:13:17.006290 19722 tablet.cc:2369] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:13:17.126391 19591 tablet.cc:2369] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:13:17.138548 19853 tablet.cc:2369] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:13:17.260259 19873 raft_consensus.cc:482] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:17.260376 19873 raft_consensus.cc:504] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:17.260735 19873 leader_election.cc:288] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3be0f5e64b374530a5ce57c4f2e2e697 (127.0.201.3:38631), 32db619b177f488cb4799e2a526fde3f (127.0.201.2:40765)
I20260619 12:13:17.263896 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:17.261029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47029 (local address 127.0.201.2:40765)
0619 12:13:17.261078 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:17.261095 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:17.261194 (+ 99us) server_negotiation.cc:407] Connection header received
0619 12:13:17.261283 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:17.261284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:17.261293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:17.261362 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:17.261584 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.262089 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.263114 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.263531 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.263622 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:17.263631 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:17.263788 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 12:13:17.263822 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:13:17.263897 19863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:17.261094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:59309 (local address 127.0.201.3:38631)
0619 12:13:17.261145 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:17.261147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:17.261157 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:13:17.261256 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:17.261258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:17.261267 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:17.261354 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:17.261513 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.262067 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.263116 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.263540 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.263622 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:17.263632 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:17.263787 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 12:13:17.263829 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:13:17.263924 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:17.260945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:38631 (local address 127.0.201.1:59309)
0619 12:13:17.261094 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:17.261106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:17.261168 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:17.261359 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:17.261360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:17.261362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:17.261443 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:17.261445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.262148 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.262149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:17.263024 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:17.263026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.263633 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.263635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:17.263701 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:17.263705 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:17.263778 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:13:17.263820 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":41,"thread_start_us":69,"threads_started":1}
I20260619 12:13:17.263944 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:17.260945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:40765 (local address 127.0.201.1:47029)
0619 12:13:17.261100 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:17.261111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:17.261212 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:17.261409 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:17.261410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:17.261412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:17.261489 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:17.261491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.262223 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.262224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:17.263014 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:17.263017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.263625 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.263626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:17.263703 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:17.263706 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:17.263781 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:13:17.263823 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260619 12:13:17.264298 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a025c7584f8c4db5bfddc0b36f50ea76" candidate_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" is_pre_election: true
I20260619 12:13:17.264439 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a025c7584f8c4db5bfddc0b36f50ea76" candidate_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db619b177f488cb4799e2a526fde3f" is_pre_election: true
I20260619 12:13:17.264488 19806 raft_consensus.cc:2454] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36bd4ecd3b8d4575a3926b68bae0dd4d in term 0.
I20260619 12:13:17.264598 19675 raft_consensus.cc:2454] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36bd4ecd3b8d4575a3926b68bae0dd4d in term 0.
I20260619 12:13:17.264725 19480 leader_election.cc:302] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36bd4ecd3b8d4575a3926b68bae0dd4d, 3be0f5e64b374530a5ce57c4f2e2e697; no voters:
I20260619 12:13:17.264855 19873 raft_consensus.cc:2781] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:13:17.264925 19873 raft_consensus.cc:482] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:13:17.264946 19873 raft_consensus.cc:3037] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:17.265617 19873 raft_consensus.cc:504] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:17.265746 19873 leader_election.cc:288] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [CANDIDATE]: Term 1 election: Requested vote from peers 3be0f5e64b374530a5ce57c4f2e2e697 (127.0.201.3:38631), 32db619b177f488cb4799e2a526fde3f (127.0.201.2:40765)
I20260619 12:13:17.265914 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a025c7584f8c4db5bfddc0b36f50ea76" candidate_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32db619b177f488cb4799e2a526fde3f"
I20260619 12:13:17.265974 19806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a025c7584f8c4db5bfddc0b36f50ea76" candidate_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3be0f5e64b374530a5ce57c4f2e2e697"
I20260619 12:13:17.266000 19675 raft_consensus.cc:3037] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:17.266063 19806 raft_consensus.cc:3037] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:17.266819 19806 raft_consensus.cc:2454] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36bd4ecd3b8d4575a3926b68bae0dd4d in term 1.
I20260619 12:13:17.266882 19675 raft_consensus.cc:2454] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36bd4ecd3b8d4575a3926b68bae0dd4d in term 1.
I20260619 12:13:17.267016 19480 leader_election.cc:302] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36bd4ecd3b8d4575a3926b68bae0dd4d, 3be0f5e64b374530a5ce57c4f2e2e697; no voters:
I20260619 12:13:17.267112 19873 raft_consensus.cc:2781] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:17.267297 19873 raft_consensus.cc:686] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [term 1 LEADER]: Becoming Leader. State: Replica: 36bd4ecd3b8d4575a3926b68bae0dd4d, State: Running, Role: LEADER
I20260619 12:13:17.267405 19873 consensus_queue.cc:237] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } }
I20260619 12:13:17.268235 19402 catalog_manager.cc:5495] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 36bd4ecd3b8d4575a3926b68bae0dd4d (127.0.201.1). New cstate: current_term: 1 leader_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36bd4ecd3b8d4575a3926b68bae0dd4d" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 43183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 } health_report { overall_health: UNKNOWN } } }
I20260619 12:13:17.358525 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:17.356077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.0.201.1:43183)
0619 12:13:17.356139 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:17.356142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:17.356153 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:13:17.356203 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:17.356204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:17.356224 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:17.356308 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:17.356471 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.356969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.357727 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:17.357931 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:17.358008 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:17.358150 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:17.358265 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:17.358379 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:17.358433 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:13:17.358445 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:13:17.360417 19806 raft_consensus.cc:1261] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 1 FOLLOWER]: Refusing update from remote peer 36bd4ecd3b8d4575a3926b68bae0dd4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:17.360411 19675 raft_consensus.cc:1261] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 1 FOLLOWER]: Refusing update from remote peer 36bd4ecd3b8d4575a3926b68bae0dd4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:17.360785 19873 consensus_queue.cc:1035] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32db619b177f488cb4799e2a526fde3f" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 40765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:17.360966 19877 consensus_queue.cc:1035] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3be0f5e64b374530a5ce57c4f2e2e697" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:17.363543 19881 mvcc.cc:205] Tried to move back new op lower bound from 7298544424385347584 to 7298544424007843840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544424385347584}]
I20260619 12:13:17.364192 19882 mvcc.cc:205] Tried to move back new op lower bound from 7298544424385347584 to 7298544424007843840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544424385347584}]
I20260619 12:13:17.365440 19401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:17.365538 19401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:17.366240 19401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ce8b7afb5614a4b90cbea8d9596657f: Sending DeleteTablet for 3 replicas of tablet a025c7584f8c4db5bfddc0b36f50ea76
I20260619 12:13:17.366577 19786 tablet_service.cc:1514] Processing DeleteTablet for tablet a025c7584f8c4db5bfddc0b36f50ea76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:17 UTC) from {username='slave'} at 127.0.0.1:48704
I20260619 12:13:17.366601 19524 tablet_service.cc:1514] Processing DeleteTablet for tablet a025c7584f8c4db5bfddc0b36f50ea76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:17 UTC) from {username='slave'} at 127.0.0.1:56514
I20260619 12:13:17.366600 19655 tablet_service.cc:1514] Processing DeleteTablet for tablet a025c7584f8c4db5bfddc0b36f50ea76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:17 UTC) from {username='slave'} at 127.0.0.1:44720
I20260619 12:13:17.366824 19890 tablet_replica.cc:330] T a025c7584f8c4db5bfddc0b36f50ea76 P 36bd4ecd3b8d4575a3926b68bae0dd4d: stopping tablet replica
I20260619 12:13:17.367007 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19462
I20260619 12:13:17.368521 19889 tablet_replica.cc:330] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: stopping tablet replica
I20260619 12:13:17.368641 19889 raft_consensus.cc:2229] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:17.368748 19889 raft_consensus.cc:2258] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:17.368891 19891 tablet_replica.cc:330] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: stopping tablet replica
I20260619 12:13:17.368971 19891 raft_consensus.cc:2229] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:17.369091 19891 raft_consensus.cc:2258] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:17.369168 19889 ts_tablet_manager.cc:1905] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:17.369503 19891 ts_tablet_manager.cc:1905] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:17.370720 19889 ts_tablet_manager.cc:1918] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:17.370805 19889 log.cc:1196] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/a025c7584f8c4db5bfddc0b36f50ea76
I20260619 12:13:17.371048 19889 ts_tablet_manager.cc:1939] T a025c7584f8c4db5bfddc0b36f50ea76 P 3be0f5e64b374530a5ce57c4f2e2e697: Deleting consensus metadata
I20260619 12:13:17.371049 19891 ts_tablet_manager.cc:1918] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:17.371119 19891 log.cc:1196] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/a025c7584f8c4db5bfddc0b36f50ea76
I20260619 12:13:17.371407 19891 ts_tablet_manager.cc:1939] T a025c7584f8c4db5bfddc0b36f50ea76 P 32db619b177f488cb4799e2a526fde3f: Deleting consensus metadata
I20260619 12:13:17.371544 19389 catalog_manager.cc:4841] TS 3be0f5e64b374530a5ce57c4f2e2e697 (127.0.201.3:38631): tablet a025c7584f8c4db5bfddc0b36f50ea76 (table test-table [id=f713a514d1e7438898bf99f60943ce5f]) successfully deleted
I20260619 12:13:17.371765 19387 catalog_manager.cc:4841] TS 32db619b177f488cb4799e2a526fde3f (127.0.201.2:40765): tablet a025c7584f8c4db5bfddc0b36f50ea76 (table test-table [id=f713a514d1e7438898bf99f60943ce5f]) successfully deleted
W20260619 12:13:17.372620 19387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:43183 (error 108)
I20260619 12:13:17.372835 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19593
I20260619 12:13:17.373495 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:17.372943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:43183 (local address 127.0.0.1:56540)
0619 12:13:17.373410 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:17.373457 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:43183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443}
W20260619 12:13:17.373768 19387 catalog_manager.cc:4584] TS 36bd4ecd3b8d4575a3926b68bae0dd4d (127.0.201.1:43183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a025c7584f8c4db5bfddc0b36f50ea76: Network error: Client connection negotiation failed: client connection to 127.0.201.1:43183: connect: Connection refused (error 111)
I20260619 12:13:17.377673 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19724
I20260619 12:13:17.382239 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19371
2026-06-19T12:13:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1086 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:17 dist-test-slave-c910 krb5kdc[19895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:17 dist-test-slave-c910 krb5kdc[19895](info): set up 2 sockets
Jun 19 12:13:17 dist-test-slave-c910 krb5kdc[19895](info): commencing operation
krb5kdc: starting...
W20260619 12:13:18.411072 804 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:18.530561 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45089
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:45089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:18.611238 19908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:18.611450 19908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:18.611474 19908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:18.612571 19908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:18.612607 19908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:18.612622 19908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:18.612634 19908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:18.612653 19908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:18.614075 19908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:45089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:18.614284 19908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:18.614468 19908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:18.619344 19913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:18.619390 19916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:18.619376 19908 server_base.cc:1034] running on GCE node
W20260619 12:13:18.619349 19914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:18.619848 19908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:18.620222 19908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:18.621413 19908 hybrid_clock.cc:648] HybridClock initialized: now 1781871198621385 us; error 44 us; skew 500 ppm
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:18.624059 19908 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:18.624565 19908 webserver.cc:460] Webserver started at http://127.0.201.62:45475/ using document root <none> and password file <none>
I20260619 12:13:18.624804 19908 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:18.624876 19908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:18.625027 19908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:18.626196 19908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "0403fcb05a66449a81bf166991162033"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "0809ac454b48fc74bc41257f49fe037b"
server_key_iv: "024bbb8d676455229e8a522bf2850865"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.626557 19908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "0403fcb05a66449a81bf166991162033"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "0809ac454b48fc74bc41257f49fe037b"
server_key_iv: "024bbb8d676455229e8a522bf2850865"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.628015 19908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260619 12:13:18.628707 19923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.628885 19908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:18.628988 19908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "0403fcb05a66449a81bf166991162033"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "0809ac454b48fc74bc41257f49fe037b"
server_key_iv: "024bbb8d676455229e8a522bf2850865"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.629061 19908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:18.636210 19908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:18.637981 19908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:18.638140 19908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:18.642175 19908 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:45089
I20260619 12:13:18.642220 19975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:45089 every 8 connection(s)
I20260619 12:13:18.642594 19908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:18.643489 19976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:18.645538 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 19908
I20260619 12:13:18.645638 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:18.645808 19976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033: Bootstrap starting.
I20260619 12:13:18.645874 804 external_mini_cluster.cc:1428] Setting key 2223866f6162d65e966b0f5563d42951
I20260619 12:13:18.646502 19976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:18.646801 19976 log.cc:824] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:18.647567 19976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033: No bootstrap required, opened a new log
I20260619 12:13:18.649657 19976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0403fcb05a66449a81bf166991162033" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45089 } }
I20260619 12:13:18.649822 19976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:18.649855 19976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0403fcb05a66449a81bf166991162033, State: Initialized, Role: FOLLOWER
I20260619 12:13:18.649976 19976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0403fcb05a66449a81bf166991162033" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45089 } }
I20260619 12:13:18.650072 19976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:18.650105 19976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:18.650161 19976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:18.650791 19976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0403fcb05a66449a81bf166991162033" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45089 } }
I20260619 12:13:18.650909 19976 leader_election.cc:302] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0403fcb05a66449a81bf166991162033; no voters:
I20260619 12:13:18.651101 19976 leader_election.cc:288] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:18.651293 19981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:18.651404 19976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:18.651472 19981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [term 1 LEADER]: Becoming Leader. State: Replica: 0403fcb05a66449a81bf166991162033, State: Running, Role: LEADER
I20260619 12:13:18.651595 19981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0403fcb05a66449a81bf166991162033" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45089 } }
I20260619 12:13:18.651983 19982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0403fcb05a66449a81bf166991162033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0403fcb05a66449a81bf166991162033" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45089 } } }
I20260619 12:13:18.651998 19983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0403fcb05a66449a81bf166991162033. Latest consensus state: current_term: 1 leader_uuid: "0403fcb05a66449a81bf166991162033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0403fcb05a66449a81bf166991162033" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45089 } } }
I20260619 12:13:18.652104 19982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:18.652128 19983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:18.652398 19987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:18.653139 19987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:18.654742 19987 catalog_manager.cc:1347] Generated new cluster ID: a3f5201b4011462dab2d49dcb5221687
I20260619 12:13:18.654791 19987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:18.657776 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:18.646391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.0.201.62:45089)
0619 12:13:18.646580 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:18.646590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:18.646627 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:13:18.647080 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:18.647087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:18.647134 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:18.647314 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:18.648064 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.648932 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.650028 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.650511 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.650598 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:18.653704 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:18.653708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:18.653708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:18.653720 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:18.656133 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:18.656513 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:18.656517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:18.656517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:18.656631 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:18.656837 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:18.656839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:18.656839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:18.657152 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:18.657317 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:18.657409 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 12:13:18.657555 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260619 12:13:18.675886 19987 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:18.676620 19987 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:18.682515 19987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033: Generated new TSK 0
I20260619 12:13:18.682718 19987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.201.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
Entry for principal kudu/127.0.201.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab.
I20260619 12:13:18.714121 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45089
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:18.798004 20002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:18.798271 20002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:18.798317 20002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:18.799546 20002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:18.799619 20002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:18.799670 20002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:13:18.801215 20002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.1.keytab
--principal=kudu/127.0.201.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:18.801493 20002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:18.801755 20002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:18.807013 20010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:18.807343 20002 server_base.cc:1034] running on GCE node
W20260619 12:13:18.807480 20008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:18.807605 20007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:18.807835 20002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:18.808099 20002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:18.809273 20002 hybrid_clock.cc:648] HybridClock initialized: now 1781871198809262 us; error 48 us; skew 500 ppm
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:18.812108 20002 init.cc:377] Logged in from keytab as kudu/127.0.201.1@KRBTEST.COM (short username kudu)
I20260619 12:13:18.812633 20002 webserver.cc:460] Webserver started at http://127.0.201.1:45239/ using document root <none> and password file <none>
I20260619 12:13:18.812860 20002 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:18.812949 20002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:18.813099 20002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:18.814158 20002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "7930f4eba44b4983a2741a6791e07453"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "64e845724dfa0e8adfed4d6702f9c029"
server_key_iv: "efcf98f69d85b1e6787b3f09b0cb921c"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.814512 20002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "7930f4eba44b4983a2741a6791e07453"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "64e845724dfa0e8adfed4d6702f9c029"
server_key_iv: "efcf98f69d85b1e6787b3f09b0cb921c"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.815956 20002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:18.816717 20017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.816906 20002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:13:18.817008 20002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "7930f4eba44b4983a2741a6791e07453"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "64e845724dfa0e8adfed4d6702f9c029"
server_key_iv: "efcf98f69d85b1e6787b3f09b0cb921c"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.817078 20002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:18.826227 20002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:18.828495 20002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:18.828655 20002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:18.828878 20002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:18.829180 20002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:18.829231 20002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.829272 20002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:18.829299 20002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.835484 20002 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:44615
I20260619 12:13:18.835532 20130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:44615 every 8 connection(s)
I20260619 12:13:18.835839 20002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:13:18.839609 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20002
I20260619 12:13:18.839776 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:13:18.840008 804 external_mini_cluster.cc:1428] Setting key 4ec26f5867d024a0f5c7674d28d3ea03
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.1@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:18.846678 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:18.836522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45089 (local address 127.0.201.1:50271)
0619 12:13:18.836667 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:18.836702 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:18.837591 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:18.837856 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:18.837860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:18.838105 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:18.838296 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:18.838301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.839005 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.839005 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:18.840031 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:18.840034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.840547 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.840548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:18.840743 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:18.841345 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:18.841356 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:18.843278 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:18.845177 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:18.845185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:18.845186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:18.845417 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:18.845644 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:18.845645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:18.845645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:18.845809 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:18.846135 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:18.846141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:18.846286 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 12:13:18.846387 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:13:18.846513 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260619 12:13:18.846675 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:18.836537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:50271 (local address 127.0.201.62:45089)
0619 12:13:18.836579 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:18.836581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:18.836793 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:13:18.837720 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:18.837721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:18.837734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:18.837801 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:18.838384 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.838921 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.840130 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.840460 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.840525 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:18.843365 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:18.843366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:18.843367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:18.843374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:18.845086 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:18.845500 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:18.845501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:18.845502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:18.845569 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:18.845883 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:18.845883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:18.845884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:18.846040 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:18.846121 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:18.846441 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 12:13:18.846537 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:13:18.847332 20131 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45089
I20260619 12:13:18.847450 20131 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:18.847652 20131 heartbeater.cc:511] Master 127.0.201.62:45089 requested a full tablet report, sending...
I20260619 12:13:18.848187 19940 ts_manager.cc:194] Registered new tserver with Master: 7930f4eba44b4983a2741a6791e07453 (127.0.201.1:44615)
I20260619 12:13:18.848831 19940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.1@KRBTEST.COM'} at 127.0.201.1:50271
WARNING: no policy specified for kudu/127.0.201.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
Entry for principal kudu/127.0.201.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab.
I20260619 12:13:18.865952 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45089
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:18.946846 20136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:18.947098 20136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:18.947137 20136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:18.948315 20136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:18.948406 20136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:18.948454 20136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:13:18.949959 20136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.2.keytab
--principal=kudu/127.0.201.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:18.950220 20136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:18.950456 20136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:18.955673 20141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:18.955724 20142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:18.955674 20144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:18.955814 20136 server_base.cc:1034] running on GCE node
I20260619 12:13:18.956156 20136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:18.956367 20136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:18.957513 20136 hybrid_clock.cc:648] HybridClock initialized: now 1781871198957502 us; error 29 us; skew 500 ppm
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:18.960274 20136 init.cc:377] Logged in from keytab as kudu/127.0.201.2@KRBTEST.COM (short username kudu)
I20260619 12:13:18.960749 20136 webserver.cc:460] Webserver started at http://127.0.201.2:38431/ using document root <none> and password file <none>
I20260619 12:13:18.960981 20136 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:18.961025 20136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:18.961189 20136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:18.962270 20136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "b96d772ba1cd429586616c5941e588ed"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "e11c347e6415af8267e7dff6be216d32"
server_key_iv: "39563b96927f3fc7c0778ba66d45f369"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.962620 20136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "b96d772ba1cd429586616c5941e588ed"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "e11c347e6415af8267e7dff6be216d32"
server_key_iv: "39563b96927f3fc7c0778ba66d45f369"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.964020 20136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:18.964745 20151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.964928 20136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:18.965018 20136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "b96d772ba1cd429586616c5941e588ed"
format_stamp: "Formatted at 2026-06-19 12:13:18 on dist-test-slave-c910"
server_key: "e11c347e6415af8267e7dff6be216d32"
server_key_iv: "39563b96927f3fc7c0778ba66d45f369"
server_key_version: "encryptionkey@0"
I20260619 12:13:18.965092 20136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:18.974942 20136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:18.976814 20136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:18.976972 20136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:18.977196 20136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:18.977504 20136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:18.977556 20136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.977596 20136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:18.977624 20136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:18.983398 20136 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:32869
I20260619 12:13:18.983451 20264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:32869 every 8 connection(s)
I20260619 12:13:18.983750 20136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
Jun 19 12:13:18 dist-test-slave-c910 krb5kdc[19895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.2@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:18.990787 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20136
I20260619 12:13:18.990923 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:13:18.991147 804 external_mini_cluster.cc:1428] Setting key cb361e544e3f85a84dcdf5dc940b4718
I20260619 12:13:18.993093 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:18.984416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:35659 (local address 127.0.201.62:45089)
0619 12:13:18.984502 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:18.984504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:18.984697 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:13:18.985621 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:18.985623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:18.985634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:18.985688 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:18.986270 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.986797 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.987850 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.988034 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.988089 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:18.990083 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:18.990085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:18.990085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:18.990092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:18.991642 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:18.992110 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:18.992111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:18.992112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:18.992173 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:18.992396 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:18.992397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:18.992398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:18.992558 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:18.992630 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:18.992861 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:13:18.992952 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:13:18.993100 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:18.984373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45089 (local address 127.0.201.2:35659)
0619 12:13:18.984546 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:18.984581 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:18.985522 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:18.985742 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:18.985745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:18.986004 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:18.986193 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:18.986197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.986877 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.986878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:18.987774 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:18.987777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:18.988107 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:18.988108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:18.988202 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:18.988651 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:18.988659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:18.989996 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:18.991715 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:18.991721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:18.991723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:18.992038 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:18.992239 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:18.992239 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:18.992240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:18.992334 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:18.992623 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:18.992626 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:18.992737 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 12:13:18.992818 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:13:18.992941 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260619 12:13:18.993964 20265 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45089
I20260619 12:13:18.994071 20265 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:18.994243 20265 heartbeater.cc:511] Master 127.0.201.62:45089 requested a full tablet report, sending...
I20260619 12:13:18.994656 19940 ts_manager.cc:194] Registered new tserver with Master: b96d772ba1cd429586616c5941e588ed (127.0.201.2:32869)
I20260619 12:13:18.995045 19940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.2@KRBTEST.COM'} at 127.0.201.2:35659
WARNING: no policy specified for kudu/127.0.201.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
Entry for principal kudu/127.0.201.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab.
I20260619 12:13:19.018373 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45089
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:19.098511 20270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:19.098759 20270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:19.098806 20270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:19.099999 20270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:19.100072 20270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:19.100117 20270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:13:19.101666 20270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/krb5kdc/kudu_127.0.201.3.keytab
--principal=kudu/127.0.201.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:45089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:19.101922 20270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:19.102141 20270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:19.107193 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:19.107159 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:19.107286 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:19.107304 20270 server_base.cc:1034] running on GCE node
I20260619 12:13:19.107750 20270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:19.108019 20270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:19.109176 20270 hybrid_clock.cc:648] HybridClock initialized: now 1781871199109158 us; error 39 us; skew 500 ppm
Jun 19 12:13:19 dist-test-slave-c910 krb5kdc[19895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:19.111639 20270 init.cc:377] Logged in from keytab as kudu/127.0.201.3@KRBTEST.COM (short username kudu)
I20260619 12:13:19.112109 20270 webserver.cc:460] Webserver started at http://127.0.201.3:45903/ using document root <none> and password file <none>
I20260619 12:13:19.112355 20270 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:19.112421 20270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:19.112598 20270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:19.113696 20270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "1b39ba05fbf445918de6a31cb881c211"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "ff129dec31d554ef1f22bc708ab0b310"
server_key_iv: "e62b51f6af815d121017b4f6f342733a"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.114073 20270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "1b39ba05fbf445918de6a31cb881c211"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "ff129dec31d554ef1f22bc708ab0b310"
server_key_iv: "e62b51f6af815d121017b4f6f342733a"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.115465 20270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:13:19.116214 20285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.116426 20270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:19.116526 20270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "1b39ba05fbf445918de6a31cb881c211"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "ff129dec31d554ef1f22bc708ab0b310"
server_key_iv: "e62b51f6af815d121017b4f6f342733a"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.116609 20270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:19.126451 20270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:19.128587 20270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:19.128770 20270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:19.129019 20270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:19.129329 20270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:19.129387 20270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.129427 20270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:19.129455 20270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.135878 20270 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:35515
I20260619 12:13:19.135923 20398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:35515 every 8 connection(s)
I20260619 12:13:19.136273 20270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
Jun 19 12:13:19 dist-test-slave-c910 krb5kdc[19895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.3@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:19.143415 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20270
I20260619 12:13:19.143533 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:13:19.143721 804 external_mini_cluster.cc:1428] Setting key d538b7c61bff7ec53508965aa09a993a
I20260619 12:13:19.146193 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.137013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:57077 (local address 127.0.201.62:45089)
0619 12:13:19.137074 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.137080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.137296 (+ 216us) server_negotiation.cc:407] Connection header received
0619 12:13:19.138201 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.138219 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.138232 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.138282 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:19.138904 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.139567 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.140588 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.140789 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.140871 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.143046 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:19.143047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:19.143047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:19.143055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:19.144859 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:19.145256 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:19.145257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:19.145258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:19.145310 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:19.145567 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:19.145568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:19.145568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:19.145683 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:19.145755 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.146024 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.146111 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:13:19.146392 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.137009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:45089 (local address 127.0.201.3:57077)
0619 12:13:19.137174 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.137214 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.138092 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.138370 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.138373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.138621 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:19.138810 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.138813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.139657 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.139658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.140474 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.140476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.140880 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.140881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.140997 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.141511 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:19.141519 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:19.142949 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:19.144945 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:19.144951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:19.144953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:19.145166 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:19.145392 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:19.145393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:19.145394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:19.145501 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:19.145758 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:19.145764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:19.145894 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.145992 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.146141 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260619 12:13:19.147946 20399 heartbeater.cc:347] Connected to a master server at 127.0.201.62:45089
I20260619 12:13:19.148054 20399 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:19.148233 20399 heartbeater.cc:511] Master 127.0.201.62:45089 requested a full tablet report, sending...
I20260619 12:13:19.148586 19940 ts_manager.cc:194] Registered new tserver with Master: 1b39ba05fbf445918de6a31cb881c211 (127.0.201.3:35515)
I20260619 12:13:19.148981 19940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.201.3@KRBTEST.COM'} at 127.0.201.3:57077
I20260619 12:13:19.155488 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:13:19.162583 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.156787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.0.201.62:45089)
0619 12:13:19.156842 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.156844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.156996 (+ 152us) server_negotiation.cc:407] Connection header received
0619 12:13:19.157061 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.157062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.157071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.157127 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:19.157770 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.158249 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.158932 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.159111 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.159173 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.160376 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:19.160377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:19.160378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:19.160383 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:19.161650 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:19.161959 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:19.161960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:19.161961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:19.162011 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:19.162177 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:19.162178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:19.162178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:19.162312 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:19.162389 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.162467 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.162522 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:13:19.164305 19940 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:13:19.165755 19940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:13:19.171828 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.167942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.0.201.3:35515)
0619 12:13:19.168143 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.168148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.168172 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:13:19.168213 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.168216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.168232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.168326 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:19.168503 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.169416 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.170679 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.171403 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.171439 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.171482 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.171696 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.171743 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260619 12:13:19.171815 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.167819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:35515 (local address 127.0.0.1:49032)
0619 12:13:19.167984 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.168028 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.168094 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.168338 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.168341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.168343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:19.168410 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.168412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.169509 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.169509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.170580 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.170583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.171529 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.171530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.171589 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.171598 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.171682 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.171739 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":58,"threads_started":1}
I20260619 12:13:19.172174 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.168002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:44615 (local address 127.0.0.1:45850)
0619 12:13:19.168372 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.168382 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.168466 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.168717 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.168718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.168720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:19.168752 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.168753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.169994 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.169995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.170759 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.170761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.171720 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.171721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.171785 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.171789 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.171854 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.171900 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260619 12:13:19.172359 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.167819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:32869 (local address 127.0.0.1:58158)
0619 12:13:19.168187 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.168196 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.168271 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.168623 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.168624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.168626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:19.168672 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.168673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.169962 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.169963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.171052 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.171054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.172124 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.172125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.172191 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.172195 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.172260 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.172296 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":28,"thread_start_us":60,"threads_started":1}
I20260619 12:13:19.172513 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.168120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.0.201.1:44615)
0619 12:13:19.168316 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.168321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.168457 (+ 136us) server_negotiation.cc:407] Connection header received
0619 12:13:19.168536 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.168539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.168553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.168629 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:19.168899 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.169897 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.170859 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.171567 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.171635 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.171685 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.172357 (+ 672us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.172404 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260619 12:13:19.172665 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.168185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.0.201.2:32869)
0619 12:13:19.168470 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.168474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.168483 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:19.168520 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.168522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.168538 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.168616 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:19.168757 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.169849 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.171152 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.172042 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.172109 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.172161 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.172535 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.172583 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260619 12:13:19.172611 20332 tablet_service.cc:1467] Processing CreateTablet for tablet d62d444514ec49568a09488856a08529 (DEFAULT_TABLE table=test-table [id=a183b9ba536046bc97fa5b016cc5bce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:19.173013 20332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d62d444514ec49568a09488856a08529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:19.173137 20065 tablet_service.cc:1467] Processing CreateTablet for tablet d62d444514ec49568a09488856a08529 (DEFAULT_TABLE table=test-table [id=a183b9ba536046bc97fa5b016cc5bce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:19.173241 20199 tablet_service.cc:1467] Processing CreateTablet for tablet d62d444514ec49568a09488856a08529 (DEFAULT_TABLE table=test-table [id=a183b9ba536046bc97fa5b016cc5bce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:19.173470 20065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d62d444514ec49568a09488856a08529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:19.173527 20199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d62d444514ec49568a09488856a08529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:19.175776 20413 tablet_bootstrap.cc:492] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Bootstrap starting.
I20260619 12:13:19.176409 20413 tablet_bootstrap.cc:654] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:19.176928 20415 tablet_bootstrap.cc:492] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: Bootstrap starting.
I20260619 12:13:19.177057 20413 log.cc:824] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:19.177436 20415 tablet_bootstrap.cc:654] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:19.177804 20413 tablet_bootstrap.cc:492] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: No bootstrap required, opened a new log
I20260619 12:13:19.177860 20413 ts_tablet_manager.cc:1397] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:13:19.177961 20415 log.cc:824] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:19.178313 20414 tablet_bootstrap.cc:492] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Bootstrap starting.
I20260619 12:13:19.178830 20414 tablet_bootstrap.cc:654] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:19.179349 20414 log.cc:824] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:19.179574 20415 tablet_bootstrap.cc:492] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: No bootstrap required, opened a new log
I20260619 12:13:19.179642 20415 ts_tablet_manager.cc:1397] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:13:19.179510 20413 raft_consensus.cc:348] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.179689 20413 raft_consensus.cc:374] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:19.179733 20413 raft_consensus.cc:729] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b39ba05fbf445918de6a31cb881c211, State: Initialized, Role: FOLLOWER
I20260619 12:13:19.179863 20413 consensus_queue.cc:260] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.180145 20413 ts_tablet_manager.cc:1428] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:13:19.180284 20399 heartbeater.cc:503] Master 127.0.201.62:45089 was elected leader, sending a full tablet report...
I20260619 12:13:19.180306 20414 tablet_bootstrap.cc:492] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: No bootstrap required, opened a new log
I20260619 12:13:19.180358 20414 ts_tablet_manager.cc:1397] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:19.181591 20415 raft_consensus.cc:348] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.181747 20415 raft_consensus.cc:374] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:19.181792 20415 raft_consensus.cc:729] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7930f4eba44b4983a2741a6791e07453, State: Initialized, Role: FOLLOWER
I20260619 12:13:19.181905 20415 consensus_queue.cc:260] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.182238 20131 heartbeater.cc:503] Master 127.0.201.62:45089 was elected leader, sending a full tablet report...
I20260619 12:13:19.182302 20415 ts_tablet_manager.cc:1428] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:19.182308 20414 raft_consensus.cc:348] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.182451 20414 raft_consensus.cc:374] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:19.182488 20414 raft_consensus.cc:729] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96d772ba1cd429586616c5941e588ed, State: Initialized, Role: FOLLOWER
I20260619 12:13:19.182610 20414 consensus_queue.cc:260] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.182991 20265 heartbeater.cc:503] Master 127.0.201.62:45089 was elected leader, sending a full tablet report...
I20260619 12:13:19.183120 20414 ts_tablet_manager.cc:1428] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 12:13:19.234349 20266 tablet.cc:2369] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:13:19.336633 20132 tablet.cc:2369] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:13:19.386884 20400 tablet.cc:2369] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:13:19.446902 20419 raft_consensus.cc:482] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:19.447095 20419 raft_consensus.cc:504] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.447533 20419 leader_election.cc:288] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b96d772ba1cd429586616c5941e588ed (127.0.201.2:32869), 7930f4eba44b4983a2741a6791e07453 (127.0.201.1:44615)
I20260619 12:13:19.450695 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.447770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:32869 (local address 127.0.201.3:36735)
0619 12:13:19.447858 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.447872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.447945 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.448244 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.448245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.448246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:19.448330 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.448332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.449040 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.449041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.449848 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.449850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.450443 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.450444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.450533 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.450537 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.450595 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.450640 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 12:13:19.450757 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.448036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:36735 (local address 127.0.201.2:32869)
0619 12:13:19.448084 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.448086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.448100 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:13:19.448133 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.448134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.448147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.448219 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:19.448410 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.448956 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.449942 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.450355 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.450409 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.450418 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.450628 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.450673 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 12:13:19.450934 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.448136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:56205 (local address 127.0.201.1:44615)
0619 12:13:19.448235 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.448237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.448245 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:19.448272 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.448272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.448283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.448347 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:19.448551 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.449086 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.450073 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.450476 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.450558 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.450569 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.450814 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.450864 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:13:19.451159 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.447935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:44615 (local address 127.0.201.3:56205)
0619 12:13:19.448152 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.448161 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.448206 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.448394 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.448395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.448398 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:19.448472 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.448474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.449169 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.449170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.449966 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.449968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.450623 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.450624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.450705 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.450709 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.451020 (+ 311us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.451074 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":136,"threads_started":1}
I20260619 12:13:19.451110 20219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62d444514ec49568a09488856a08529" candidate_uuid: "1b39ba05fbf445918de6a31cb881c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96d772ba1cd429586616c5941e588ed" is_pre_election: true
I20260619 12:13:19.451308 20219 raft_consensus.cc:2454] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b39ba05fbf445918de6a31cb881c211 in term 0.
I20260619 12:13:19.451547 20287 leader_election.cc:302] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b39ba05fbf445918de6a31cb881c211, b96d772ba1cd429586616c5941e588ed; no voters:
I20260619 12:13:19.451668 20085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62d444514ec49568a09488856a08529" candidate_uuid: "1b39ba05fbf445918de6a31cb881c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7930f4eba44b4983a2741a6791e07453" is_pre_election: true
I20260619 12:13:19.451676 20419 raft_consensus.cc:2781] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:13:19.451756 20419 raft_consensus.cc:482] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:13:19.451781 20419 raft_consensus.cc:3037] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:19.451809 20085 raft_consensus.cc:2454] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b39ba05fbf445918de6a31cb881c211 in term 0.
I20260619 12:13:19.452616 20419 raft_consensus.cc:504] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.452739 20419 leader_election.cc:288] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [CANDIDATE]: Term 1 election: Requested vote from peers b96d772ba1cd429586616c5941e588ed (127.0.201.2:32869), 7930f4eba44b4983a2741a6791e07453 (127.0.201.1:44615)
I20260619 12:13:19.452915 20085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62d444514ec49568a09488856a08529" candidate_uuid: "1b39ba05fbf445918de6a31cb881c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7930f4eba44b4983a2741a6791e07453"
I20260619 12:13:19.452910 20219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d62d444514ec49568a09488856a08529" candidate_uuid: "1b39ba05fbf445918de6a31cb881c211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b96d772ba1cd429586616c5941e588ed"
I20260619 12:13:19.452994 20219 raft_consensus.cc:3037] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:19.453017 20085 raft_consensus.cc:3037] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:19.453732 20085 raft_consensus.cc:2454] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b39ba05fbf445918de6a31cb881c211 in term 1.
I20260619 12:13:19.453732 20219 raft_consensus.cc:2454] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b39ba05fbf445918de6a31cb881c211 in term 1.
I20260619 12:13:19.453938 20287 leader_election.cc:302] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b39ba05fbf445918de6a31cb881c211, 7930f4eba44b4983a2741a6791e07453; no voters:
I20260619 12:13:19.454042 20419 raft_consensus.cc:2781] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:19.454212 20419 raft_consensus.cc:686] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 1 LEADER]: Becoming Leader. State: Replica: 1b39ba05fbf445918de6a31cb881c211, State: Running, Role: LEADER
I20260619 12:13:19.454336 20419 consensus_queue.cc:237] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } }
I20260619 12:13:19.455140 19939 catalog_manager.cc:5495] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b39ba05fbf445918de6a31cb881c211 (127.0.201.3). New cstate: current_term: 1 leader_uuid: "1b39ba05fbf445918de6a31cb881c211" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b39ba05fbf445918de6a31cb881c211" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35515 } health_report { overall_health: HEALTHY } } }
I20260619 12:13:19.529397 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.526793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.0.201.3:35515)
0619 12:13:19.526875 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.526884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.526892 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:19.526925 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.526926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.526945 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.527026 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:19.527166 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.527767 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.528594 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.528790 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.528860 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.529041 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:19.529141 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:19.529237 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.529256 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.529275 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:13:19.531517 20219 raft_consensus.cc:1261] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 1 FOLLOWER]: Refusing update from remote peer 1b39ba05fbf445918de6a31cb881c211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:19.531589 20085 raft_consensus.cc:1261] T d62d444514ec49568a09488856a08529 P 7930f4eba44b4983a2741a6791e07453 [term 1 FOLLOWER]: Refusing update from remote peer 1b39ba05fbf445918de6a31cb881c211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:19.531941 20423 consensus_queue.cc:1035] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7930f4eba44b4983a2741a6791e07453" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 44615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:19.532063 20419 consensus_queue.cc:1035] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b96d772ba1cd429586616c5941e588ed" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 32869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:19.535351 20427 mvcc.cc:205] Tried to move back new op lower bound from 7298544433277845504 to 7298544432965488640. Current Snapshot: MvccSnapshot[applied={T|T < 7298544433277845504 or (T in {7298544433277845504})}]
I20260619 12:13:19.536648 19940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:19.536753 19940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:19.537484 19940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0403fcb05a66449a81bf166991162033: Sending DeleteTablet for 3 replicas of tablet d62d444514ec49568a09488856a08529
I20260619 12:13:19.537801 20199 tablet_service.cc:1514] Processing DeleteTablet for tablet d62d444514ec49568a09488856a08529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:19 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:58158
I20260619 12:13:19.537827 20065 tablet_service.cc:1514] Processing DeleteTablet for tablet d62d444514ec49568a09488856a08529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:19 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:45850
I20260619 12:13:19.538009 20332 tablet_service.cc:1514] Processing DeleteTablet for tablet d62d444514ec49568a09488856a08529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:19 UTC) from {username='kudu', principal='kudu/127.0.201.62@KRBTEST.COM'} at 127.0.0.1:49032
I20260619 12:13:19.538213 20436 tablet_replica.cc:330] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: stopping tablet replica
I20260619 12:13:19.538348 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20002
I20260619 12:13:19.540848 20438 tablet_replica.cc:330] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: stopping tablet replica
I20260619 12:13:19.540956 20438 raft_consensus.cc:2229] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:13:19.541079 20438 raft_consensus.cc:2258] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:19.541204 20436 raft_consensus.cc:2229] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:19.541335 20436 raft_consensus.cc:2258] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:19.541757 20438 ts_tablet_manager.cc:1905] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:19.541954 20436 ts_tablet_manager.cc:1905] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:19.543485 20438 ts_tablet_manager.cc:1918] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:19.543563 20438 log.cc:1196] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/d62d444514ec49568a09488856a08529
I20260619 12:13:19.543566 20436 ts_tablet_manager.cc:1918] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:19.543628 20436 log.cc:1196] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/d62d444514ec49568a09488856a08529
I20260619 12:13:19.543843 20438 ts_tablet_manager.cc:1939] T d62d444514ec49568a09488856a08529 P 1b39ba05fbf445918de6a31cb881c211: Deleting consensus metadata
I20260619 12:13:19.543883 20436 ts_tablet_manager.cc:1939] T d62d444514ec49568a09488856a08529 P b96d772ba1cd429586616c5941e588ed: Deleting consensus metadata
I20260619 12:13:19.544272 19927 catalog_manager.cc:4841] TS 1b39ba05fbf445918de6a31cb881c211 (127.0.201.3:35515): tablet d62d444514ec49568a09488856a08529 (table test-table [id=a183b9ba536046bc97fa5b016cc5bce2]) successfully deleted
I20260619 12:13:19.544394 19925 catalog_manager.cc:4841] TS b96d772ba1cd429586616c5941e588ed (127.0.201.2:32869): tablet d62d444514ec49568a09488856a08529 (table test-table [id=a183b9ba536046bc97fa5b016cc5bce2]) successfully deleted
W20260619 12:13:19.546708 19925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:44615 (error 108)
I20260619 12:13:19.546980 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20136
I20260619 12:13:19.547181 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.547055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:44615 (local address 127.0.0.1:45860)
0619 12:13:19.547103 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.547143 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 12:13:19.547309 19925 catalog_manager.cc:4584] TS 7930f4eba44b4983a2741a6791e07453 (127.0.201.1:44615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d62d444514ec49568a09488856a08529: Network error: Client connection negotiation failed: client connection to 127.0.201.1:44615: connect: Connection refused (error 111)
I20260619 12:13:19.552413 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20270
I20260619 12:13:19.557374 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 19908
2026-06-19T12:13:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2175 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T12:13:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:19Z Disabled control of system clock
I20260619 12:13:19.584595 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42037
--webserver_interface=127.0.201.62
--webserver_port=0
--builtin_ntp_servers=127.0.201.20:39863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:19.666364 20443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:19.666560 20443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:19.666580 20443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:19.667709 20443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:19.667748 20443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:19.667760 20443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:19.667773 20443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:19.667784 20443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:19.669037 20443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:19.669212 20443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:19.669418 20443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:19.673960 20448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:19.674010 20449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:19.673944 20451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:19.674293 20443 server_base.cc:1034] running on GCE node
I20260619 12:13:19.674510 20443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:19.674767 20443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:19.675904 20443 hybrid_clock.cc:648] HybridClock initialized: now 1781871199675895 us; error 30 us; skew 500 ppm
I20260619 12:13:19.677141 20443 webserver.cc:460] Webserver started at http://127.0.201.62:37597/ using document root <none> and password file <none>
I20260619 12:13:19.677403 20443 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:19.677453 20443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:19.677625 20443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:19.678615 20443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "92a2f047694b492f81c125cbc572977e"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "d894b36656b874910c91b9ca9fa8800f"
server_key_iv: "5aa13e45b75a6d1bfdcd0f96bae1513a"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.678944 20443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "92a2f047694b492f81c125cbc572977e"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "d894b36656b874910c91b9ca9fa8800f"
server_key_iv: "5aa13e45b75a6d1bfdcd0f96bae1513a"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.680299 20443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:19.681046 20457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.681278 20443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:19.681366 20443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "92a2f047694b492f81c125cbc572977e"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "d894b36656b874910c91b9ca9fa8800f"
server_key_iv: "5aa13e45b75a6d1bfdcd0f96bae1513a"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.681442 20443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:19.709625 20443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:19.710016 20443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:19.710170 20443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:19.714643 20443 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42037
I20260619 12:13:19.714710 20509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42037 every 8 connection(s)
I20260619 12:13:19.715025 20443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:19.716022 20510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:19.718268 20510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e: Bootstrap starting.
I20260619 12:13:19.718730 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20443
I20260619 12:13:19.718835 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:19.718986 20510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:19.719065 804 external_mini_cluster.cc:1428] Setting key f2be994c7c925ebb26bb93e0b582aa25
I20260619 12:13:19.719352 20510 log.cc:824] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:19.720299 20510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e: No bootstrap required, opened a new log
I20260619 12:13:19.721639 20510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a2f047694b492f81c125cbc572977e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42037 } }
I20260619 12:13:19.721765 20510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:19.721786 20510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a2f047694b492f81c125cbc572977e, State: Initialized, Role: FOLLOWER
I20260619 12:13:19.721925 20510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a2f047694b492f81c125cbc572977e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42037 } }
I20260619 12:13:19.721994 20510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:19.722028 20510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:19.722077 20510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:19.722586 20510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a2f047694b492f81c125cbc572977e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42037 } }
I20260619 12:13:19.722671 20510 leader_election.cc:302] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92a2f047694b492f81c125cbc572977e; no voters:
I20260619 12:13:19.722863 20510 leader_election.cc:288] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:19.722890 20515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:19.723001 20515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [term 1 LEADER]: Becoming Leader. State: Replica: 92a2f047694b492f81c125cbc572977e, State: Running, Role: LEADER
I20260619 12:13:19.723084 20515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a2f047694b492f81c125cbc572977e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42037 } }
I20260619 12:13:19.723142 20510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:19.723414 20516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92a2f047694b492f81c125cbc572977e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a2f047694b492f81c125cbc572977e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42037 } } }
I20260619 12:13:19.723624 20516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:19.723567 20517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a2f047694b492f81c125cbc572977e. Latest consensus state: current_term: 1 leader_uuid: "92a2f047694b492f81c125cbc572977e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a2f047694b492f81c125cbc572977e" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42037 } } }
I20260619 12:13:19.723855 20517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:19.724040 20523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:19.724551 20523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:19.725788 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.719610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.0.201.62:42037)
0619 12:13:19.719836 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.719846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.719878 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:13:19.720381 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.720390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.720462 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.720609 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:19.720822 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.722312 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.723358 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.724015 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.724113 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.725080 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:19.725085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:19.725085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:19.725286 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:19.725330 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:19.725345 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:19.725420 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.725504 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.725577 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260619 12:13:19.726199 20523 catalog_manager.cc:1347] Generated new cluster ID: cd655f8d31d64f08a678b08688a2de34
I20260619 12:13:19.726251 20523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:19.747114 20523 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:19.747684 20523 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:19.752588 20523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e: Generated new TSK 0
I20260619 12:13:19.752748 20523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:19.755659 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.1:0
--local_ip_for_outbound_sockets=127.0.201.1
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42037
--builtin_ntp_servers=127.0.201.20:39863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:13:19.837263 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:19.837496 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:19.837530 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:19.838635 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:19.838697 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:19.838738 20534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.1
I20260619 12:13:19.840099 20534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.201.1
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.1
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:19.840346 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:19.840628 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:19.846230 20540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:19.846238 20542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:19.846462 20534 server_base.cc:1034] running on GCE node
W20260619 12:13:19.846239 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:19.846768 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:19.847002 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:19.848168 20534 hybrid_clock.cc:648] HybridClock initialized: now 1781871199848161 us; error 41 us; skew 500 ppm
I20260619 12:13:19.849365 20534 webserver.cc:460] Webserver started at http://127.0.201.1:36761/ using document root <none> and password file <none>
I20260619 12:13:19.849607 20534 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:19.849674 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:19.849831 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:19.850832 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data/instance:
uuid: "6be322153e8643daa9b1c55e9060123f"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "73ecd3d279ae25c19867e3e64b6f0197"
server_key_iv: "d8de9b73373ac804109ccd48eaa75c38"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.851186 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal/instance:
uuid: "6be322153e8643daa9b1c55e9060123f"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "73ecd3d279ae25c19867e3e64b6f0197"
server_key_iv: "d8de9b73373ac804109ccd48eaa75c38"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.852486 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:19.853238 20548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.853411 20534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:19.853506 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal
uuid: "6be322153e8643daa9b1c55e9060123f"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "73ecd3d279ae25c19867e3e64b6f0197"
server_key_iv: "d8de9b73373ac804109ccd48eaa75c38"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.853587 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:19.870997 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:19.871380 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:19.871541 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:19.871776 20534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:19.872104 20534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:19.872157 20534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.872196 20534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:19.872223 20534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.878089 20534 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.1:37217
I20260619 12:13:19.878135 20661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.1:37217 every 8 connection(s)
I20260619 12:13:19.878458 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/data/info.pb
I20260619 12:13:19.880455 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20534
I20260619 12:13:19.880571 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-0/wal/instance
I20260619 12:13:19.880753 804 external_mini_cluster.cc:1428] Setting key 59c6f9f853840febb24dc9cc61452bbd
I20260619 12:13:19.881744 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.2:0
--local_ip_for_outbound_sockets=127.0.201.2
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42037
--builtin_ntp_servers=127.0.201.20:39863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:13:19.884397 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.879115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:55541 (local address 127.0.201.62:42037)
0619 12:13:19.879160 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.879162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.879430 (+ 268us) server_negotiation.cc:407] Connection header received
0619 12:13:19.879920 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.879921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.879931 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.879982 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:19.880715 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.881373 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.882862 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.883051 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.883134 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.884078 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:19.884079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:19.884080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:19.884084 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:19.884102 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:19.884107 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:19.884169 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.884245 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.884270 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:13:19.884547 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.879098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42037 (local address 127.0.201.1:55541)
0619 12:13:19.879306 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.879340 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.879822 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.880034 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.880037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.880045 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:19.880606 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.880614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.881476 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.881477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.882765 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.882768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.883142 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.883143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.883325 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.883909 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:19.883910 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:19.883944 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:19.883966 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:19.884171 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:19.884174 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:19.884176 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.884250 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.884342 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260619 12:13:19.885027 20662 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42037
I20260619 12:13:19.885155 20662 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:19.885354 20662 heartbeater.cc:511] Master 127.0.201.62:42037 requested a full tablet report, sending...
I20260619 12:13:19.885869 20474 ts_manager.cc:194] Registered new tserver with Master: 6be322153e8643daa9b1c55e9060123f (127.0.201.1:37217)
I20260619 12:13:19.886489 20474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.1:55541
W20260619 12:13:19.962246 20665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:19.962435 20665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:19.962456 20665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:19.963650 20665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:19.963687 20665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:19.963732 20665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.2
I20260619 12:13:19.965014 20665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.201.2
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.2
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:19.965238 20665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:19.965512 20665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:19.970369 20670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:19.970343 20673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:19.970381 20665 server_base.cc:1034] running on GCE node
W20260619 12:13:19.970348 20671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:19.970737 20665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:19.970984 20665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:19.972158 20665 hybrid_clock.cc:648] HybridClock initialized: now 1781871199972141 us; error 29 us; skew 500 ppm
I20260619 12:13:19.973487 20665 webserver.cc:460] Webserver started at http://127.0.201.2:39415/ using document root <none> and password file <none>
I20260619 12:13:19.973728 20665 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:19.973798 20665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:19.973938 20665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:19.974947 20665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data/instance:
uuid: "a2a08b7616fe4d04ac644e58a9d9c932"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "4338aa5844d1a671a931a3e5c2eb2821"
server_key_iv: "6397947d01f2d70500e41d17163a9ef2"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.975342 20665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal/instance:
uuid: "a2a08b7616fe4d04ac644e58a9d9c932"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "4338aa5844d1a671a931a3e5c2eb2821"
server_key_iv: "6397947d01f2d70500e41d17163a9ef2"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.976686 20665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:19.977367 20679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.977545 20665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:19.977656 20665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal
uuid: "a2a08b7616fe4d04ac644e58a9d9c932"
format_stamp: "Formatted at 2026-06-19 12:13:19 on dist-test-slave-c910"
server_key: "4338aa5844d1a671a931a3e5c2eb2821"
server_key_iv: "6397947d01f2d70500e41d17163a9ef2"
server_key_version: "encryptionkey@0"
I20260619 12:13:19.977738 20665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:19.986143 20665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:19.986404 20665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:19.986548 20665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:19.986753 20665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:19.987071 20665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:19.987126 20665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.987164 20665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:19.987210 20665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:19.992712 20665 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.2:45065
I20260619 12:13:19.992748 20792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.2:45065 every 8 connection(s)
I20260619 12:13:19.993053 20665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/data/info.pb
I20260619 12:13:19.996925 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20665
I20260619 12:13:19.997030 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal/instance
I20260619 12:13:19.997267 804 external_mini_cluster.cc:1428] Setting key 691280726efb8c5b831b89cfe8c1020b
I20260619 12:13:19.998085 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.993791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.2:48453 (local address 127.0.201.62:42037)
0619 12:13:19.993848 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:19.993850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:19.994054 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:13:19.994485 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:19.994485 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:19.994495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:19.994546 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:19.995148 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.995686 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.996607 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.996799 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.996847 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.997832 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:19.997833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:19.997834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:19.997838 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:19.997853 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:19.997857 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:19.997936 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:19.998012 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 12:13:19.998031 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:13:19.998243 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.201.3:0
--local_ip_for_outbound_sockets=127.0.201.3
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42037
--builtin_ntp_servers=127.0.201.20:39863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:13:19.998545 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:19.993788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42037 (local address 127.0.201.2:48453)
0619 12:13:19.993945 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:19.993973 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:19.994409 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:19.994592 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:19.994600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:19.994607 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:19.995083 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.995086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.995760 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.995761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.996538 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:19.996540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:19.996881 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:19.996882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:19.997046 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:19.997689 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:19.997690 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:19.997715 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:19.997735 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:19.997939 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:19.997944 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:19.997947 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:19.998014 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:13:19.998116 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260619 12:13:19.999003 20793 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42037
I20260619 12:13:19.999090 20793 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:19.999287 20793 heartbeater.cc:511] Master 127.0.201.62:42037 requested a full tablet report, sending...
I20260619 12:13:19.999615 20474 ts_manager.cc:194] Registered new tserver with Master: a2a08b7616fe4d04ac644e58a9d9c932 (127.0.201.2:45065)
I20260619 12:13:19.999979 20474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.2:48453
W20260619 12:13:20.076066 20796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:20.076244 20796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:20.076264 20796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:20.077345 20796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:20.077382 20796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:20.077431 20796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.201.3
I20260619 12:13:20.078675 20796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:39863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.201.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.201.3
--webserver_port=0
--tserver_master_addrs=127.0.201.62:42037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.201.3
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:20.078919 20796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:20.079186 20796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:20.084126 20804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:20.084133 20802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:20.084261 20801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:20.084371 20796 server_base.cc:1034] running on GCE node
I20260619 12:13:20.084550 20796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:20.084767 20796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:20.085922 20796 hybrid_clock.cc:648] HybridClock initialized: now 1781871200085910 us; error 43 us; skew 500 ppm
I20260619 12:13:20.087041 20796 webserver.cc:460] Webserver started at http://127.0.201.3:37931/ using document root <none> and password file <none>
I20260619 12:13:20.087374 20796 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:20.087447 20796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:20.087579 20796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:20.088567 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data/instance:
uuid: "0a7adc8358a24303a54893916beaccc2"
format_stamp: "Formatted at 2026-06-19 12:13:20 on dist-test-slave-c910"
server_key: "f1d39c1a6799781a1ca4063c86553503"
server_key_iv: "e823fca3224a4279cf297bd81a117fbf"
server_key_version: "encryptionkey@0"
I20260619 12:13:20.088910 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal/instance:
uuid: "0a7adc8358a24303a54893916beaccc2"
format_stamp: "Formatted at 2026-06-19 12:13:20 on dist-test-slave-c910"
server_key: "f1d39c1a6799781a1ca4063c86553503"
server_key_iv: "e823fca3224a4279cf297bd81a117fbf"
server_key_version: "encryptionkey@0"
I20260619 12:13:20.090143 20796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:20.090782 20810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:20.090945 20796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:20.091041 20796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal
uuid: "0a7adc8358a24303a54893916beaccc2"
format_stamp: "Formatted at 2026-06-19 12:13:20 on dist-test-slave-c910"
server_key: "f1d39c1a6799781a1ca4063c86553503"
server_key_iv: "e823fca3224a4279cf297bd81a117fbf"
server_key_version: "encryptionkey@0"
I20260619 12:13:20.091125 20796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:20.109031 20796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:20.109301 20796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:20.109442 20796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:20.109643 20796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:13:20.109931 20796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:13:20.110003 20796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:20.110037 20796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:13:20.110064 20796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:20.115716 20796 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.3:35565
I20260619 12:13:20.115760 20923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.3:35565 every 8 connection(s)
I20260619 12:13:20.116037 20796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/data/info.pb
I20260619 12:13:20.120621 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.116624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.3:60085 (local address 127.0.201.62:42037)
0619 12:13:20.116668 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.116670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.116938 (+ 268us) server_negotiation.cc:407] Connection header received
0619 12:13:20.117343 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.117344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.117353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.117409 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:20.118002 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.118463 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.119430 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.119605 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.119658 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.120386 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:20.120387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:20.120387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:20.120392 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:20.120404 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:20.120408 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:20.120454 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.120546 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.120565 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 12:13:20.120859 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.116598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42037 (local address 127.0.201.3:60085)
0619 12:13:20.116813 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.116857 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:20.117264 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:20.117481 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:20.117484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:20.117491 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:20.117934 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.117937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.118536 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.118537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.119360 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.119362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.119675 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.119676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.119793 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.120255 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:13:20.120255 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:20.120285 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:13:20.120307 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:20.120481 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:20.120484 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:20.120487 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:13:20.120536 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 12:13:20.120605 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 12:13:20.121349 20924 heartbeater.cc:347] Connected to a master server at 127.0.201.62:42037
I20260619 12:13:20.121444 20924 heartbeater.cc:464] Registering TS with master...
I20260619 12:13:20.121603 20924 heartbeater.cc:511] Master 127.0.201.62:42037 requested a full tablet report, sending...
I20260619 12:13:20.121901 20474 ts_manager.cc:194] Registered new tserver with Master: 0a7adc8358a24303a54893916beaccc2 (127.0.201.3:35565)
I20260619 12:13:20.122208 20474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.201.3:60085
I20260619 12:13:20.123098 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20796
I20260619 12:13:20.123226 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal/instance
I20260619 12:13:20.123394 804 external_mini_cluster.cc:1428] Setting key dbf9b6304db35230368e2c16ac7f1f29
I20260619 12:13:20.124289 804 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:13:20.128082 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.125551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.0.201.62:42037)
0619 12:13:20.125602 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.125603 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.125704 (+ 101us) server_negotiation.cc:407] Connection header received
0619 12:13:20.125772 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.125773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.125780 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.125825 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:20.125935 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.126363 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.127091 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.127323 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.127368 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.127921 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:20.127922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:20.127922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:20.127926 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:20.127936 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:20.127940 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:20.127985 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.128020 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.128036 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:13:20.129688 20474 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:13:20.131201 20474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:13:20.137193 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.133180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:37217 (local address 127.0.0.1:60808)
0619 12:13:20.133615 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.133624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:20.133671 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:20.133880 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:20.133881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:20.133884 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:20.133939 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.133941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.135143 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.135144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.136010 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.136013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.136896 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.136897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.136967 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.136979 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 12:13:20.137051 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:13:20.137117 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260619 12:13:20.137321 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.133155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:45065 (local address 127.0.0.1:35576)
0619 12:13:20.133333 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.133370 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:20.133431 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:20.133659 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:20.133661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:20.133663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:20.133723 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.133725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.134990 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.134991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.136239 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.136242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.137051 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.137052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.137129 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.137133 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:20.137188 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:13:20.137229 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":62,"threads_started":1}
I20260619 12:13:20.137410 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.133382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:35565 (local address 127.0.0.1:44306)
0619 12:13:20.133957 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.133966 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:20.134012 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:20.134288 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:20.134289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:20.134290 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:20.134318 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.134319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.135165 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.135166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.136419 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.136420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.137182 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.137183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.137242 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.137246 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:13:20.137299 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:13:20.137340 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":26,"thread_start_us":88,"threads_started":1}
I20260619 12:13:20.137485 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.133441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.0.201.1:37217)
0619 12:13:20.133585 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.133589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.133724 (+ 135us) server_negotiation.cc:407] Connection header received
0619 12:13:20.133767 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.133769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.133785 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.133866 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:20.134034 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.135068 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.136108 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.136800 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.136878 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.136975 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.137357 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.137396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260619 12:13:20.137790 20934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.133250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.0.201.2:45065)
0619 12:13:20.133434 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.133439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.133453 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:13:20.133517 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.133519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.133536 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.133650 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:20.133784 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.134889 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.136318 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.136977 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.137038 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.137134 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.137201 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.137708 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260619 12:13:20.137802 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.133571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.0.201.3:35565)
0619 12:13:20.133852 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.133856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.134104 (+ 248us) server_negotiation.cc:407] Connection header received
0619 12:13:20.134140 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.134142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.134156 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.134218 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:20.134392 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.135081 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.136491 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.137004 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.137058 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.137148 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.137489 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.137730 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":73,"threads_started":1}
I20260619 12:13:20.138131 20596 tablet_service.cc:1467] Processing CreateTablet for tablet d39cc31ad2d7470ab3a2559b094061c1 (DEFAULT_TABLE table=test-table [id=86cbf1cfc5e54ee9b99a317e664390af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:20.138307 20858 tablet_service.cc:1467] Processing CreateTablet for tablet d39cc31ad2d7470ab3a2559b094061c1 (DEFAULT_TABLE table=test-table [id=86cbf1cfc5e54ee9b99a317e664390af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:20.138476 20596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d39cc31ad2d7470ab3a2559b094061c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:20.138594 20858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d39cc31ad2d7470ab3a2559b094061c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:20.141669 20939 tablet_bootstrap.cc:492] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Bootstrap starting.
I20260619 12:13:20.141911 20938 tablet_bootstrap.cc:492] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f: Bootstrap starting.
I20260619 12:13:20.141991 20727 tablet_service.cc:1467] Processing CreateTablet for tablet d39cc31ad2d7470ab3a2559b094061c1 (DEFAULT_TABLE table=test-table [id=86cbf1cfc5e54ee9b99a317e664390af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:13:20.142324 20727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d39cc31ad2d7470ab3a2559b094061c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:20.142552 20938 tablet_bootstrap.cc:654] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:20.142825 20938 log.cc:824] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:20.143528 20939 tablet_bootstrap.cc:654] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:20.143621 20938 tablet_bootstrap.cc:492] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f: No bootstrap required, opened a new log
I20260619 12:13:20.143740 20938 ts_tablet_manager.cc:1397] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:20.143929 20939 log.cc:824] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:20.144752 20939 tablet_bootstrap.cc:492] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: No bootstrap required, opened a new log
I20260619 12:13:20.144829 20939 ts_tablet_manager.cc:1397] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:13:20.145602 20942 tablet_bootstrap.cc:492] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Bootstrap starting.
I20260619 12:13:20.146077 20938 raft_consensus.cc:348] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.146196 20938 raft_consensus.cc:374] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:20.146219 20938 raft_consensus.cc:729] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be322153e8643daa9b1c55e9060123f, State: Initialized, Role: FOLLOWER
I20260619 12:13:20.146330 20938 consensus_queue.cc:260] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.146399 20942 tablet_bootstrap.cc:654] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:20.146587 20938 ts_tablet_manager.cc:1428] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:20.146621 20662 heartbeater.cc:503] Master 127.0.201.62:42037 was elected leader, sending a full tablet report...
I20260619 12:13:20.146878 20942 log.cc:824] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:20.147665 20942 tablet_bootstrap.cc:492] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: No bootstrap required, opened a new log
I20260619 12:13:20.147735 20942 ts_tablet_manager.cc:1397] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:13:20.147796 20939 raft_consensus.cc:348] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.147971 20939 raft_consensus.cc:374] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:20.148012 20939 raft_consensus.cc:729] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7adc8358a24303a54893916beaccc2, State: Initialized, Role: FOLLOWER
I20260619 12:13:20.148120 20939 consensus_queue.cc:260] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.148370 20939 ts_tablet_manager.cc:1428] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 12:13:20.148404 20924 heartbeater.cc:503] Master 127.0.201.62:42037 was elected leader, sending a full tablet report...
I20260619 12:13:20.149997 20942 raft_consensus.cc:348] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.150112 20942 raft_consensus.cc:374] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:20.150132 20942 raft_consensus.cc:729] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a08b7616fe4d04ac644e58a9d9c932, State: Initialized, Role: FOLLOWER
I20260619 12:13:20.150264 20942 consensus_queue.cc:260] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.150497 20942 ts_tablet_manager.cc:1428] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 12:13:20.150542 20793 heartbeater.cc:503] Master 127.0.201.62:42037 was elected leader, sending a full tablet report...
I20260619 12:13:20.176777 20943 raft_consensus.cc:482] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:13:20.176872 20943 raft_consensus.cc:504] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.177220 20943 leader_election.cc:288] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a08b7616fe4d04ac644e58a9d9c932 (127.0.201.2:45065), 0a7adc8358a24303a54893916beaccc2 (127.0.201.3:35565)
I20260619 12:13:20.180907 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.177574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:59421 (local address 127.0.201.3:35565)
0619 12:13:20.177627 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.177629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.177731 (+ 102us) server_negotiation.cc:407] Connection header received
0619 12:13:20.177796 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.177797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.177806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.177851 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:20.178016 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.178673 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.179972 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.180540 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.180612 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.180621 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.180794 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.180833 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:13:20.180899 20934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.177502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.201.1:47163 (local address 127.0.201.2:45065)
0619 12:13:20.177560 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.177561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.177570 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:20.177596 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.177597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.177606 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.177666 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:13:20.177829 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.178601 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.179972 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.180540 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.180612 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.180621 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.180794 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.180833 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:13:20.180931 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.177402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.2:45065 (local address 127.0.201.1:47163)
0619 12:13:20.177477 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.177488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:20.177529 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:20.177687 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:20.177688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:20.177691 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:20.177763 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.177765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.178672 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.178673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.179881 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.179884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.180623 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.180624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.180706 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.180713 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:13:20.180774 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:13:20.180818 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":12}
I20260619 12:13:20.180944 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.177506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.3:35565 (local address 127.0.201.1:59421)
0619 12:13:20.177678 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.177687 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:20.177736 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:20.177853 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:20.177854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:20.177855 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:13:20.177932 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.177934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.178746 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.178746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.179888 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:20.179889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.180625 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.180625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:20.180707 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.180713 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:13:20.180774 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:13:20.180817 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":6,"thread_start_us":94,"threads_started":1}
I20260619 12:13:20.181420 20878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39cc31ad2d7470ab3a2559b094061c1" candidate_uuid: "6be322153e8643daa9b1c55e9060123f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a7adc8358a24303a54893916beaccc2" is_pre_election: true
I20260619 12:13:20.181454 20747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39cc31ad2d7470ab3a2559b094061c1" candidate_uuid: "6be322153e8643daa9b1c55e9060123f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" is_pre_election: true
I20260619 12:13:20.181566 20878 raft_consensus.cc:2454] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6be322153e8643daa9b1c55e9060123f in term 0.
I20260619 12:13:20.181591 20747 raft_consensus.cc:2454] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6be322153e8643daa9b1c55e9060123f in term 0.
I20260619 12:13:20.181821 20552 leader_election.cc:302] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a7adc8358a24303a54893916beaccc2, 6be322153e8643daa9b1c55e9060123f; no voters:
I20260619 12:13:20.181978 20943 raft_consensus.cc:2781] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:13:20.182042 20943 raft_consensus.cc:482] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:13:20.182060 20943 raft_consensus.cc:3037] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:20.182718 20943 raft_consensus.cc:504] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.182838 20943 leader_election.cc:288] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [CANDIDATE]: Term 1 election: Requested vote from peers a2a08b7616fe4d04ac644e58a9d9c932 (127.0.201.2:45065), 0a7adc8358a24303a54893916beaccc2 (127.0.201.3:35565)
I20260619 12:13:20.183007 20747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39cc31ad2d7470ab3a2559b094061c1" candidate_uuid: "6be322153e8643daa9b1c55e9060123f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a08b7616fe4d04ac644e58a9d9c932"
I20260619 12:13:20.183025 20878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39cc31ad2d7470ab3a2559b094061c1" candidate_uuid: "6be322153e8643daa9b1c55e9060123f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a7adc8358a24303a54893916beaccc2"
I20260619 12:13:20.183107 20878 raft_consensus.cc:3037] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:20.183107 20747 raft_consensus.cc:3037] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:20.183997 20747 raft_consensus.cc:2454] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6be322153e8643daa9b1c55e9060123f in term 1.
I20260619 12:13:20.183997 20878 raft_consensus.cc:2454] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6be322153e8643daa9b1c55e9060123f in term 1.
I20260619 12:13:20.184168 20552 leader_election.cc:302] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a7adc8358a24303a54893916beaccc2, 6be322153e8643daa9b1c55e9060123f; no voters:
I20260619 12:13:20.184258 20943 raft_consensus.cc:2781] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:20.184422 20943 raft_consensus.cc:686] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [term 1 LEADER]: Becoming Leader. State: Replica: 6be322153e8643daa9b1c55e9060123f, State: Running, Role: LEADER
I20260619 12:13:20.184520 20943 consensus_queue.cc:237] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } }
I20260619 12:13:20.185226 20474 catalog_manager.cc:5495] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6be322153e8643daa9b1c55e9060123f (127.0.201.1). New cstate: current_term: 1 leader_uuid: "6be322153e8643daa9b1c55e9060123f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6be322153e8643daa9b1c55e9060123f" member_type: VOTER last_known_addr { host: "127.0.201.1" port: 37217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 } health_report { overall_health: UNKNOWN } } }
I20260619 12:13:20.193284 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.191187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.0.201.1:37217)
0619 12:13:20.191259 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:20.191261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:20.191294 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:13:20.191322 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:20.191323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:20.191340 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:20.191431 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:20.191564 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.191986 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.192662 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:20.192832 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:20.192878 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:20.193012 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:20.193094 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:20.193154 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:20.193184 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 12:13:20.193199 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:13:20.194895 20878 raft_consensus.cc:1261] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 1 FOLLOWER]: Refusing update from remote peer 6be322153e8643daa9b1c55e9060123f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:20.194901 20747 raft_consensus.cc:1261] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 1 FOLLOWER]: Refusing update from remote peer 6be322153e8643daa9b1c55e9060123f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:20.195254 20948 consensus_queue.cc:1035] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a7adc8358a24303a54893916beaccc2" member_type: VOTER last_known_addr { host: "127.0.201.3" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:20.195374 20943 consensus_queue.cc:1035] T d39cc31ad2d7470ab3a2559b094061c1 P 6be322153e8643daa9b1c55e9060123f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a08b7616fe4d04ac644e58a9d9c932" member_type: VOTER last_known_addr { host: "127.0.201.2" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:20.198215 20953 mvcc.cc:205] Tried to move back new op lower bound from 7298544435995975680 to 7298544435956449280. Current Snapshot: MvccSnapshot[applied={T|T < 7298544435995975680}]
I20260619 12:13:20.199661 20474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:20.199750 20474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:13:20.200358 20474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92a2f047694b492f81c125cbc572977e: Sending DeleteTablet for 3 replicas of tablet d39cc31ad2d7470ab3a2559b094061c1
I20260619 12:13:20.200789 20596 tablet_service.cc:1514] Processing DeleteTablet for tablet d39cc31ad2d7470ab3a2559b094061c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:20 UTC) from {username='slave'} at 127.0.0.1:60808
I20260619 12:13:20.200810 20858 tablet_service.cc:1514] Processing DeleteTablet for tablet d39cc31ad2d7470ab3a2559b094061c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:20 UTC) from {username='slave'} at 127.0.0.1:44306
I20260619 12:13:20.200997 20727 tablet_service.cc:1514] Processing DeleteTablet for tablet d39cc31ad2d7470ab3a2559b094061c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:13:20 UTC) from {username='slave'} at 127.0.0.1:35576
I20260619 12:13:20.201181 20962 tablet_replica.cc:330] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: stopping tablet replica
I20260619 12:13:20.201270 20962 raft_consensus.cc:2229] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:20.201364 20962 raft_consensus.cc:2258] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:20.201452 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20534
I20260619 12:13:20.202579 20962 ts_tablet_manager.cc:1905] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:20.204280 20962 ts_tablet_manager.cc:1918] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:20.204362 20962 log.cc:1196] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-1/wal/wals/d39cc31ad2d7470ab3a2559b094061c1
I20260619 12:13:20.204592 20962 ts_tablet_manager.cc:1939] T d39cc31ad2d7470ab3a2559b094061c1 P a2a08b7616fe4d04ac644e58a9d9c932: Deleting consensus metadata
I20260619 12:13:20.204969 20461 catalog_manager.cc:4841] TS a2a08b7616fe4d04ac644e58a9d9c932 (127.0.201.2:45065): tablet d39cc31ad2d7470ab3a2559b094061c1 (table test-table [id=86cbf1cfc5e54ee9b99a317e664390af]) successfully deleted
W20260619 12:13:20.206864 20459 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.201.1:37217 (error 108)
I20260619 12:13:20.206923 20960 tablet_replica.cc:330] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: stopping tablet replica
I20260619 12:13:20.207019 20960 raft_consensus.cc:2229] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:13:20.207113 20960 raft_consensus.cc:2258] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:13:20.207127 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20665
I20260619 12:13:20.207525 20960 ts_tablet_manager.cc:1905] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:13:20.208775 20960 ts_tablet_manager.cc:1918] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:13:20.208848 20960 log.cc:1196] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Deleting WAL directory at /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781870963166527-804-0/minicluster-data/ts-2/wal/wals/d39cc31ad2d7470ab3a2559b094061c1
I20260619 12:13:20.209110 20960 ts_tablet_manager.cc:1939] T d39cc31ad2d7470ab3a2559b094061c1 P 0a7adc8358a24303a54893916beaccc2: Deleting consensus metadata
I20260619 12:13:20.209498 20461 catalog_manager.cc:4841] TS 0a7adc8358a24303a54893916beaccc2 (127.0.201.3:35565): tablet d39cc31ad2d7470ab3a2559b094061c1 (table test-table [id=86cbf1cfc5e54ee9b99a317e664390af]) successfully deleted
I20260619 12:13:20.211453 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:20.211327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.1:37217 (local address 127.0.0.1:60834)
0619 12:13:20.211383 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:20.211422 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.1:37217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:13:20.211558 20459 catalog_manager.cc:4584] TS 6be322153e8643daa9b1c55e9060123f (127.0.201.1:37217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d39cc31ad2d7470ab3a2559b094061c1: Network error: Client connection negotiation failed: client connection to 127.0.201.1:37217: connect: Connection refused (error 111)
I20260619 12:13:20.212126 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20796
I20260619 12:13:20.216828 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20443
2026-06-19T12:13:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (658 ms)
[----------] 6 tests from EncryptionPolicyTest (8873 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:20 dist-test-slave-c910 krb5kdc[20966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:20 dist-test-slave-c910 krb5kdc[20966](info): set up 2 sockets
Jun 19 12:13:20 dist-test-slave-c910 krb5kdc[20966](info): commencing operation
krb5kdc: starting...
W20260619 12:13:21.251092 804 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:21 dist-test-slave-c910 krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:21.362407 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:21.441219 20979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:21.441447 20979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:21.441485 20979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:21.442605 20979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:21.442670 20979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:21.442696 20979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:21.442734 20979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:21.442761 20979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:21.444211 20979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45899
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:21.444480 20979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:21.444710 20979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:21.449331 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:21.449389 20987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:21.449331 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:21.449409 20979 server_base.cc:1034] running on GCE node
I20260619 12:13:21.449848 20979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:21.450117 20979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:21.451275 20979 hybrid_clock.cc:648] HybridClock initialized: now 1781871201451259 us; error 34 us; skew 500 ppm
Jun 19 12:13:21 dist-test-slave-c910 krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:21.457984 20979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:21.458436 20979 webserver.cc:460] Webserver started at http://127.0.0.1:41375/ using document root <none> and password file <none>
I20260619 12:13:21.458652 20979 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:21.458729 20979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:21.458876 20979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:21.459934 20979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "7ebee9504564485bb0b61ca053cdfa7d"
format_stamp: "Formatted at 2026-06-19 12:13:21 on dist-test-slave-c910"
server_key: "c49e9f0ffcad404a573d2386ee0cd02a"
server_key_iv: "6623c40afd416544252eaf306e80ab02"
server_key_version: "encryptionkey@0"
I20260619 12:13:21.460284 20979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "7ebee9504564485bb0b61ca053cdfa7d"
format_stamp: "Formatted at 2026-06-19 12:13:21 on dist-test-slave-c910"
server_key: "c49e9f0ffcad404a573d2386ee0cd02a"
server_key_iv: "6623c40afd416544252eaf306e80ab02"
server_key_version: "encryptionkey@0"
I20260619 12:13:21.461567 20979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:13:21.462280 20994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:21.462445 20979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:21.462548 20979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "7ebee9504564485bb0b61ca053cdfa7d"
format_stamp: "Formatted at 2026-06-19 12:13:21 on dist-test-slave-c910"
server_key: "c49e9f0ffcad404a573d2386ee0cd02a"
server_key_iv: "6623c40afd416544252eaf306e80ab02"
server_key_version: "encryptionkey@0"
I20260619 12:13:21.462625 20979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:21.470482 20979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:21.472750 20979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:21.472903 20979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:21.476912 20979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45899
I20260619 12:13:21.476940 21046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45899 every 8 connection(s)
I20260619 12:13:21.477253 20979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:21.478025 21047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:21.480132 21047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d: Bootstrap starting.
I20260619 12:13:21.480649 21047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:21.480870 21047 log.cc:824] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:21.481524 21047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d: No bootstrap required, opened a new log
I20260619 12:13:21.482973 21047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebee9504564485bb0b61ca053cdfa7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45899 } }
I20260619 12:13:21.483103 21047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:21.483124 21047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ebee9504564485bb0b61ca053cdfa7d, State: Initialized, Role: FOLLOWER
I20260619 12:13:21.483284 21047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebee9504564485bb0b61ca053cdfa7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45899 } }
I20260619 12:13:21.483379 21047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:21.483419 21047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:21.483465 21047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:21.483974 21047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebee9504564485bb0b61ca053cdfa7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45899 } }
I20260619 12:13:21.484052 21047 leader_election.cc:302] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ebee9504564485bb0b61ca053cdfa7d; no voters:
I20260619 12:13:21.484248 21047 leader_election.cc:288] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:21.484344 21050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:21.484508 21047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:21.484516 21050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [term 1 LEADER]: Becoming Leader. State: Replica: 7ebee9504564485bb0b61ca053cdfa7d, State: Running, Role: LEADER
I20260619 12:13:21.484658 21050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebee9504564485bb0b61ca053cdfa7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45899 } }
I20260619 12:13:21.485419 21052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ebee9504564485bb0b61ca053cdfa7d. Latest consensus state: current_term: 1 leader_uuid: "7ebee9504564485bb0b61ca053cdfa7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebee9504564485bb0b61ca053cdfa7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45899 } } }
I20260619 12:13:21.485529 21052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:21.485596 21051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ebee9504564485bb0b61ca053cdfa7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebee9504564485bb0b61ca053cdfa7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45899 } } }
I20260619 12:13:21.485663 21051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:21.485809 21063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:21.486486 21063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:21.486809 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 20979
I20260619 12:13:21.486937 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:21.487141 804 external_mini_cluster.cc:1428] Setting key eeb4b525d6876a607d1709acc426fa00
I20260619 12:13:21.488534 21063 catalog_manager.cc:1347] Generated new cluster ID: b9e8348c11854801b3268c28cc50836a
I20260619 12:13:21.488585 21063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:13:21 dist-test-slave-c910 krb5kdc[20966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:13:21.496093 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:21.487605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39349 (local address 127.0.0.1:45899)
0619 12:13:21.487779 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:21.487788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:21.487828 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:13:21.487938 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:21.487945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:21.487996 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:21.488135 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:21.488791 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:21.489689 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:21.490460 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:21.490869 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:21.490992 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:21.493147 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:21.493150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:21.493151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:21.493162 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:21.494592 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:21.494964 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:21.494967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:21.494967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:21.495068 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:21.495292 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:21.495292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:21.495293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:21.495516 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:21.495689 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:21.495791 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:13:21.495901 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260619 12:13:21.497113 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:21.501996 21063 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:21.502640 21063 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:21.504470 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:21.498366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49475 (local address 127.0.0.1:45899)
0619 12:13:21.498404 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:21.498406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:21.498529 (+ 123us) server_negotiation.cc:407] Connection header received
0619 12:13:21.498597 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:21.498597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:21.498606 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:21.498659 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:21.499152 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:21.499668 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:21.500350 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:21.500572 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:21.500648 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:21.501945 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:21.501946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:21.501947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:21.501953 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:21.503359 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:21.503704 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:21.503705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:21.503705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:21.503756 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:21.503978 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:21.503979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:21.503979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:21.504084 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:21.504188 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:21.504309 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 12:13:21.504370 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:21.517009 21063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ebee9504564485bb0b61ca053cdfa7d: Generated new TSK 0
I20260619 12:13:21.517180 21063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:21.520810 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 20979
W20260619 12:13:21.525581 20964 connection.cc:535] client connection to 127.0.0.1:45899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45899): Connection reset by peer (error 104)
2026-06-19T12:13:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:21 dist-test-slave-c910 krb5kdc[21078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:21 dist-test-slave-c910 krb5kdc[21078](info): set up 2 sockets
Jun 19 12:13:21 dist-test-slave-c910 krb5kdc[21078](info): commencing operation
krb5kdc: starting...
W20260619 12:13:22.547251 804 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:22 dist-test-slave-c910 krb5kdc[21078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:22.658289 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:22.737214 21091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:22.737473 21091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:22.737511 21091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:22.738703 21091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:22.738767 21091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:22.738793 21091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:22.738826 21091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:22.738850 21091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:22.740320 21091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:22.740586 21091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:22.740816 21091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:22.745764 21097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:22.745786 21096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:22.745788 21099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:22.745805 21091 server_base.cc:1034] running on GCE node
I20260619 12:13:22.746196 21091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:22.746500 21091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:22.747663 21091 hybrid_clock.cc:648] HybridClock initialized: now 1781871202747648 us; error 27 us; skew 500 ppm
Jun 19 12:13:22 dist-test-slave-c910 krb5kdc[21078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:22.750007 21091 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:22.750449 21091 webserver.cc:460] Webserver started at http://127.0.0.1:39841/ using document root <none> and password file <none>
I20260619 12:13:22.750666 21091 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:22.750731 21091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:22.750859 21091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:22.751892 21091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "d03394227ce24d71a5706a3d9eec5b3c"
format_stamp: "Formatted at 2026-06-19 12:13:22 on dist-test-slave-c910"
server_key: "95d4709ad38d215f1cb618abf5d60d74"
server_key_iv: "a587ce001b4461010bf703f7b17fb43c"
server_key_version: "encryptionkey@0"
I20260619 12:13:22.752218 21091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "d03394227ce24d71a5706a3d9eec5b3c"
format_stamp: "Formatted at 2026-06-19 12:13:22 on dist-test-slave-c910"
server_key: "95d4709ad38d215f1cb618abf5d60d74"
server_key_iv: "a587ce001b4461010bf703f7b17fb43c"
server_key_version: "encryptionkey@0"
I20260619 12:13:22.753510 21091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:13:22.754217 21106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:22.754413 21091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:22.754505 21091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "d03394227ce24d71a5706a3d9eec5b3c"
format_stamp: "Formatted at 2026-06-19 12:13:22 on dist-test-slave-c910"
server_key: "95d4709ad38d215f1cb618abf5d60d74"
server_key_iv: "a587ce001b4461010bf703f7b17fb43c"
server_key_version: "encryptionkey@0"
I20260619 12:13:22.754581 21091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:22.764792 21091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:22.766659 21091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:22.766817 21091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:22.770723 21091 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34267
I20260619 12:13:22.770757 21158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34267 every 8 connection(s)
I20260619 12:13:22.771087 21091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:22.771965 21159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:22.772187 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21091
I20260619 12:13:22.772321 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:22.772531 804 external_mini_cluster.cc:1428] Setting key bffe5ab0f9a70b75369c3281dffc275e
I20260619 12:13:22.775118 21159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c: Bootstrap starting.
I20260619 12:13:22.775811 21159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:22.776108 21159 log.cc:824] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:22.776799 21159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c: No bootstrap required, opened a new log
I20260619 12:13:22.777757 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:22.773032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40959 (local address 127.0.0.1:34267)
0619 12:13:22.773196 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:22.773203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:22.773226 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:13:22.773638 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:22.773644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:22.773687 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:22.773793 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:22.774021 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:22.774844 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:22.775979 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:22.776403 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:22.776523 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:22.777205 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:22.777209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:22.777209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:22.777369 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:22.777397 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:22.777407 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:22.777501 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:22.777578 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 12:13:22.777604 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260619 12:13:22.778851 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:22.779172 21159 raft_consensus.cc:348] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d03394227ce24d71a5706a3d9eec5b3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260619 12:13:22.779394 21159 raft_consensus.cc:374] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:22.779440 21159 raft_consensus.cc:729] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d03394227ce24d71a5706a3d9eec5b3c, State: Initialized, Role: FOLLOWER
I20260619 12:13:22.779556 21159 consensus_queue.cc:260] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d03394227ce24d71a5706a3d9eec5b3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260619 12:13:22.779639 21159 raft_consensus.cc:388] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:22.779693 21159 raft_consensus.cc:482] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:22.779740 21159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:22.780311 21159 raft_consensus.cc:504] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d03394227ce24d71a5706a3d9eec5b3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260619 12:13:22.780428 21159 leader_election.cc:302] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d03394227ce24d71a5706a3d9eec5b3c; no voters:
I20260619 12:13:22.780606 21159 leader_election.cc:288] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:22.780642 21169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:22.780810 21169 raft_consensus.cc:686] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [term 1 LEADER]: Becoming Leader. State: Replica: d03394227ce24d71a5706a3d9eec5b3c, State: Running, Role: LEADER
I20260619 12:13:22.780876 21159 sys_catalog.cc:564] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:22.780946 21169 consensus_queue.cc:237] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d03394227ce24d71a5706a3d9eec5b3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260619 12:13:22.781250 21170 sys_catalog.cc:455] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d03394227ce24d71a5706a3d9eec5b3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d03394227ce24d71a5706a3d9eec5b3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } } }
I20260619 12:13:22.781327 21170 sys_catalog.cc:458] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:22.781237 21169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d03394227ce24d71a5706a3d9eec5b3c. Latest consensus state: current_term: 1 leader_uuid: "d03394227ce24d71a5706a3d9eec5b3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d03394227ce24d71a5706a3d9eec5b3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } } }
I20260619 12:13:22.781524 21169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:22.781643 21175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:22.782084 21175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:22.783812 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:22.780306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38499 (local address 127.0.0.1:34267)
0619 12:13:22.780341 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:22.780342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:22.780403 (+ 61us) server_negotiation.cc:407] Connection header received
0619 12:13:22.780478 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:22.780479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:22.780487 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:22.780547 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:22.780720 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:22.781453 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:22.782446 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:22.782725 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:22.782817 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:22.783516 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:22.783518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:22.783518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:22.783523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:22.783538 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:22.783542 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:22.783593 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:22.783710 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:13:22.783720 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:13:22.784101 21175 catalog_manager.cc:1347] Generated new cluster ID: 2a44f32117304900b62fdbe00390a85f
I20260619 12:13:22.784152 21175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:22.791493 21175 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:22.791978 21175 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:22.797662 21175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d03394227ce24d71a5706a3d9eec5b3c: Generated new TSK 0
I20260619 12:13:22.797828 21175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:22.803054 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21091
W20260619 12:13:22.806931 21076 connection.cc:535] client connection to 127.0.0.1:34267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34267): Connection reset by peer (error 104)
2026-06-19T12:13:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:22 dist-test-slave-c910 krb5kdc[21190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:22 dist-test-slave-c910 krb5kdc[21190](info): set up 2 sockets
Jun 19 12:13:22 dist-test-slave-c910 krb5kdc[21190](info): commencing operation
krb5kdc: starting...
W20260619 12:13:23.823489 804 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:23 dist-test-slave-c910 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:23.934459 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:24.012547 21203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:24.012777 21203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:24.012814 21203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:24.013949 21203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:24.014019 21203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:24.014043 21203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:24.014065 21203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:24.014093 21203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:24.015539 21203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43021
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:24.015787 21203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:24.016014 21203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:24.020747 21211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:24.020702 21209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:24.020721 21208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:24.020907 21203 server_base.cc:1034] running on GCE node
I20260619 12:13:24.021238 21203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:24.021530 21203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:24.022687 21203 hybrid_clock.cc:648] HybridClock initialized: now 1781871204022674 us; error 28 us; skew 500 ppm
Jun 19 12:13:24 dist-test-slave-c910 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:24.025411 21203 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:24.025961 21203 webserver.cc:460] Webserver started at http://127.0.0.1:43523/ using document root <none> and password file <none>
I20260619 12:13:24.026265 21203 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:24.026327 21203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:24.026474 21203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:24.027643 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "546cd875707e4227ad7176089f0cecee"
format_stamp: "Formatted at 2026-06-19 12:13:24 on dist-test-slave-c910"
server_key: "5595aa9ec93a7ee52c17f3f494936c25"
server_key_iv: "8b44e0a58a937f800151bbd6b6a04869"
server_key_version: "encryptionkey@0"
I20260619 12:13:24.028013 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "546cd875707e4227ad7176089f0cecee"
format_stamp: "Formatted at 2026-06-19 12:13:24 on dist-test-slave-c910"
server_key: "5595aa9ec93a7ee52c17f3f494936c25"
server_key_iv: "8b44e0a58a937f800151bbd6b6a04869"
server_key_version: "encryptionkey@0"
I20260619 12:13:24.029366 21203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 12:13:24.030079 21218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:24.030251 21203 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:24.030349 21203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "546cd875707e4227ad7176089f0cecee"
format_stamp: "Formatted at 2026-06-19 12:13:24 on dist-test-slave-c910"
server_key: "5595aa9ec93a7ee52c17f3f494936c25"
server_key_iv: "8b44e0a58a937f800151bbd6b6a04869"
server_key_version: "encryptionkey@0"
I20260619 12:13:24.030439 21203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:24.033793 21203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:24.036123 21203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:24.036280 21203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:24.040154 21203 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43021
I20260619 12:13:24.040184 21270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43021 every 8 connection(s)
I20260619 12:13:24.040508 21203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:24.041388 21271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:24.043548 21271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee: Bootstrap starting.
I20260619 12:13:24.044081 21271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:24.044365 21271 log.cc:824] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:24.044998 21271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee: No bootstrap required, opened a new log
I20260619 12:13:24.046494 21271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd875707e4227ad7176089f0cecee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43021 } }
I20260619 12:13:24.046617 21271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:24.046636 21271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546cd875707e4227ad7176089f0cecee, State: Initialized, Role: FOLLOWER
I20260619 12:13:24.046797 21271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd875707e4227ad7176089f0cecee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43021 } }
I20260619 12:13:24.046857 21271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:24.046878 21271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:24.046940 21271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:24.047497 21271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd875707e4227ad7176089f0cecee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43021 } }
I20260619 12:13:24.047602 21271 leader_election.cc:302] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 546cd875707e4227ad7176089f0cecee; no voters:
I20260619 12:13:24.047748 21271 leader_election.cc:288] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:24.047813 21274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:24.048081 21271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:24.048216 21274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [term 1 LEADER]: Becoming Leader. State: Replica: 546cd875707e4227ad7176089f0cecee, State: Running, Role: LEADER
I20260619 12:13:24.048298 21274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd875707e4227ad7176089f0cecee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43021 } }
I20260619 12:13:24.048667 21275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "546cd875707e4227ad7176089f0cecee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd875707e4227ad7176089f0cecee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43021 } } }
I20260619 12:13:24.048748 21275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:24.048781 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21203
I20260619 12:13:24.048902 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:24.048959 21280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:24.049168 804 external_mini_cluster.cc:1428] Setting key 7fbf80b4e31054cf063dd9debeb9460f
I20260619 12:13:24.049240 21276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 546cd875707e4227ad7176089f0cecee. Latest consensus state: current_term: 1 leader_uuid: "546cd875707e4227ad7176089f0cecee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd875707e4227ad7176089f0cecee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43021 } } }
I20260619 12:13:24.049299 21276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:24.049494 21280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:24.051525 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:24.049574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53589 (local address 127.0.0.1:43021)
0619 12:13:24.049751 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:24.049760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:24.049945 (+ 185us) server_negotiation.cc:407] Connection header received
0619 12:13:24.050119 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:24.050128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:24.050264 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:24.050412 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:24.051070 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:24.051073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:24.051073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:24.051077 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:24.051120 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:24.051133 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:24.051240 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:24.051324 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 12:13:24.051361 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260619 12:13:24.051597 21280 catalog_manager.cc:1347] Generated new cluster ID: c1f79003aec846fa996120582057be4d
I20260619 12:13:24.051671 21280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:24.052376 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:24.054816 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:24.053889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56025 (local address 127.0.0.1:43021)
0619 12:13:24.053920 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:24.053922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:24.053973 (+ 51us) server_negotiation.cc:407] Connection header received
0619 12:13:24.054026 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:24.054027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:24.054031 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:24.054095 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:24.054618 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:24.054619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:24.054619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:24.054623 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:24.054635 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:24.054639 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:24.054703 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:24.054743 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 12:13:24.054756 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:13:24.068022 21280 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:24.068183 21280 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:24.071472 21280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 546cd875707e4227ad7176089f0cecee: Generated new TSK 0
I20260619 12:13:24.071645 21280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:24.079180 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21203
2026-06-19T12:13:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:24 dist-test-slave-c910 krb5kdc[21302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:24 dist-test-slave-c910 krb5kdc[21302](info): set up 2 sockets
Jun 19 12:13:24 dist-test-slave-c910 krb5kdc[21302](info): commencing operation
krb5kdc: starting...
W20260619 12:13:25.111362 804 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:25 dist-test-slave-c910 krb5kdc[21302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:25.225531 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:25.306118 21315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:25.306313 21315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:25.306334 21315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:25.307518 21315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:25.307557 21315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:25.307570 21315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:25.307583 21315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:25.307619 21315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:25.309548 21315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:25.309813 21315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:25.310047 21315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:25.314996 21315 server_base.cc:1034] running on GCE node
W20260619 12:13:25.314963 21320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:25.314944 21323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:25.314925 21321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:25.315486 21315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:25.315809 21315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:25.316992 21315 hybrid_clock.cc:648] HybridClock initialized: now 1781871205316980 us; error 48 us; skew 500 ppm
Jun 19 12:13:25 dist-test-slave-c910 krb5kdc[21302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871205, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:25.319432 21315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:25.319857 21315 webserver.cc:460] Webserver started at http://127.0.0.1:33171/ using document root <none> and password file <none>
I20260619 12:13:25.320051 21315 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:25.320091 21315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:25.320175 21315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:25.321156 21315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "e5b36dbd818541ceb4211bc362ab3537"
format_stamp: "Formatted at 2026-06-19 12:13:25 on dist-test-slave-c910"
server_key: "07535b21154bd565dfb31ff5a65ba9b1"
server_key_iv: "7ce16fd37b22e2873300d6767605c736"
server_key_version: "encryptionkey@0"
I20260619 12:13:25.321432 21315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "e5b36dbd818541ceb4211bc362ab3537"
format_stamp: "Formatted at 2026-06-19 12:13:25 on dist-test-slave-c910"
server_key: "07535b21154bd565dfb31ff5a65ba9b1"
server_key_iv: "7ce16fd37b22e2873300d6767605c736"
server_key_version: "encryptionkey@0"
I20260619 12:13:25.322618 21315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:13:25.323336 21330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:25.323508 21315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:13:25.323606 21315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "e5b36dbd818541ceb4211bc362ab3537"
format_stamp: "Formatted at 2026-06-19 12:13:25 on dist-test-slave-c910"
server_key: "07535b21154bd565dfb31ff5a65ba9b1"
server_key_iv: "7ce16fd37b22e2873300d6767605c736"
server_key_version: "encryptionkey@0"
I20260619 12:13:25.323691 21315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:25.337832 21315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:25.339591 21315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:25.339768 21315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:25.343629 21315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42593
I20260619 12:13:25.343663 21382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42593 every 8 connection(s)
I20260619 12:13:25.344019 21315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:25.344851 21383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:25.347025 21383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537: Bootstrap starting.
I20260619 12:13:25.347595 21383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:25.347879 21383 log.cc:824] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:25.348531 21383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537: No bootstrap required, opened a new log
I20260619 12:13:25.349877 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21315
I20260619 12:13:25.350000 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:25.350189 804 external_mini_cluster.cc:1428] Setting key 2d79710b3f61ff4ff59935df8c71839b
I20260619 12:13:25.350397 21383 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b36dbd818541ceb4211bc362ab3537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260619 12:13:25.350561 21383 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:25.350600 21383 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b36dbd818541ceb4211bc362ab3537, State: Initialized, Role: FOLLOWER
I20260619 12:13:25.350757 21383 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b36dbd818541ceb4211bc362ab3537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260619 12:13:25.350879 21383 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:25.350947 21383 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:25.351003 21383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:25.352162 21383 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b36dbd818541ceb4211bc362ab3537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260619 12:13:25.352290 21383 leader_election.cc:302] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5b36dbd818541ceb4211bc362ab3537; no voters:
I20260619 12:13:25.352502 21383 leader_election.cc:288] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:25.352533 21388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:25.352633 21388 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [term 1 LEADER]: Becoming Leader. State: Replica: e5b36dbd818541ceb4211bc362ab3537, State: Running, Role: LEADER
I20260619 12:13:25.352761 21383 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:25.352768 21388 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b36dbd818541ceb4211bc362ab3537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } }
I20260619 12:13:25.353176 21388 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5b36dbd818541ceb4211bc362ab3537. Latest consensus state: current_term: 1 leader_uuid: "e5b36dbd818541ceb4211bc362ab3537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b36dbd818541ceb4211bc362ab3537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } } }
I20260619 12:13:25.353287 21388 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:25.353447 21389 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5b36dbd818541ceb4211bc362ab3537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b36dbd818541ceb4211bc362ab3537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42593 } } }
I20260619 12:13:25.353559 21389 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:25.354029 21397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:25.354544 21397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:25.356199 21397 catalog_manager.cc:1347] Generated new cluster ID: d6245548d27a479e918f1ab8565f6430
I20260619 12:13:25.356262 21397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:13:25 dist-test-slave-c910 krb5kdc[21302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:13:25.361233 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:25.350678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48717 (local address 127.0.0.1:42593)
0619 12:13:25.350828 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:25.350838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:25.350888 (+ 50us) server_negotiation.cc:407] Connection header received
0619 12:13:25.351041 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:25.351047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:25.351115 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:25.351289 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:25.352056 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:25.353433 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:25.354497 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:25.355260 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:25.355362 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:25.357728 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:25.357732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:25.357732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:25.357744 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:25.359447 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:25.359822 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:25.359825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:25.359825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:25.359926 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:25.360125 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:25.360126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:25.360126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:25.360358 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:25.360953 (+ 595us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:25.361007 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:13:25.361105 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260619 12:13:25.362066 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:25.369236 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:25.363311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40347 (local address 127.0.0.1:42593)
0619 12:13:25.363514 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:25.363515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:25.363541 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:13:25.363608 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:25.363609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:25.363619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:25.363674 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:25.364148 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:25.364611 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:25.365331 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:25.365548 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:25.365645 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:25.366773 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:25.366775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:25.366775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:25.366781 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:25.368156 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:25.368516 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:25.368517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:25.368517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:25.368566 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:25.368759 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:25.368760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:25.368760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:25.368870 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:25.368970 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:25.369068 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:13:25.369138 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260619 12:13:25.370517 21397 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:25.371187 21397 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:25.385838 21397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5b36dbd818541ceb4211bc362ab3537: Generated new TSK 0
I20260619 12:13:25.386006 21397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:25.393102 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21315
2026-06-19T12:13:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:25 dist-test-slave-c910 krb5kdc[21414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:25 dist-test-slave-c910 krb5kdc[21414](info): set up 2 sockets
Jun 19 12:13:25 dist-test-slave-c910 krb5kdc[21414](info): commencing operation
krb5kdc: starting...
W20260619 12:13:26.435155 804 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:26 dist-test-slave-c910 krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:26.549578 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:26.627250 21427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:26.627436 21427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:26.627458 21427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:26.628574 21427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:26.628613 21427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:26.628628 21427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:26.628639 21427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:26.628651 21427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:26.630074 21427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:26.630309 21427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:26.630527 21427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:26.635093 21435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:26.635080 21433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:26.635070 21432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:26.635314 21427 server_base.cc:1034] running on GCE node
I20260619 12:13:26.635517 21427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:26.635730 21427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:26.636868 21427 hybrid_clock.cc:648] HybridClock initialized: now 1781871206636861 us; error 32 us; skew 500 ppm
Jun 19 12:13:26 dist-test-slave-c910 krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:26.645581 21427 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:26.646024 21427 webserver.cc:460] Webserver started at http://127.0.0.1:42121/ using document root <none> and password file <none>
I20260619 12:13:26.646252 21427 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:26.646297 21427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:26.646421 21427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:26.647462 21427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "d64369ae444744eaaebbcb85a422c619"
format_stamp: "Formatted at 2026-06-19 12:13:26 on dist-test-slave-c910"
server_key: "766a8ae427878925da8173efba11d326"
server_key_iv: "e8e3f4bc7c5d8157127d3538da98cd4d"
server_key_version: "encryptionkey@0"
I20260619 12:13:26.647799 21427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "d64369ae444744eaaebbcb85a422c619"
format_stamp: "Formatted at 2026-06-19 12:13:26 on dist-test-slave-c910"
server_key: "766a8ae427878925da8173efba11d326"
server_key_iv: "e8e3f4bc7c5d8157127d3538da98cd4d"
server_key_version: "encryptionkey@0"
I20260619 12:13:26.649036 21427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:13:26.649713 21442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:26.649883 21427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:26.649979 21427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "d64369ae444744eaaebbcb85a422c619"
format_stamp: "Formatted at 2026-06-19 12:13:26 on dist-test-slave-c910"
server_key: "766a8ae427878925da8173efba11d326"
server_key_iv: "e8e3f4bc7c5d8157127d3538da98cd4d"
server_key_version: "encryptionkey@0"
I20260619 12:13:26.650053 21427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:26.669963 21427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:26.671800 21427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:26.671962 21427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:26.675923 21427 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39463
I20260619 12:13:26.675976 21494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39463 every 8 connection(s)
I20260619 12:13:26.676316 21427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:26.677127 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:26.679241 21495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619: Bootstrap starting.
I20260619 12:13:26.679761 21495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:26.679980 21495 log.cc:824] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:26.680560 21495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619: No bootstrap required, opened a new log
I20260619 12:13:26.682040 21495 raft_consensus.cc:348] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64369ae444744eaaebbcb85a422c619" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39463 } }
I20260619 12:13:26.682169 21495 raft_consensus.cc:374] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:26.682191 21495 raft_consensus.cc:729] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64369ae444744eaaebbcb85a422c619, State: Initialized, Role: FOLLOWER
I20260619 12:13:26.682286 21495 consensus_queue.cc:260] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64369ae444744eaaebbcb85a422c619" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39463 } }
I20260619 12:13:26.682343 21495 raft_consensus.cc:388] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:26.682363 21495 raft_consensus.cc:482] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:26.682387 21495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:26.682827 21495 raft_consensus.cc:504] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64369ae444744eaaebbcb85a422c619" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39463 } }
I20260619 12:13:26.682905 21495 leader_election.cc:302] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d64369ae444744eaaebbcb85a422c619; no voters:
I20260619 12:13:26.683015 21495 leader_election.cc:288] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:26.683081 21498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:26.683269 21498 raft_consensus.cc:686] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [term 1 LEADER]: Becoming Leader. State: Replica: d64369ae444744eaaebbcb85a422c619, State: Running, Role: LEADER
I20260619 12:13:26.683300 21495 sys_catalog.cc:564] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:26.683362 21498 consensus_queue.cc:237] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64369ae444744eaaebbcb85a422c619" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39463 } }
I20260619 12:13:26.683779 21500 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d64369ae444744eaaebbcb85a422c619. Latest consensus state: current_term: 1 leader_uuid: "d64369ae444744eaaebbcb85a422c619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64369ae444744eaaebbcb85a422c619" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39463 } } }
I20260619 12:13:26.683916 21500 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:26.684079 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21427
I20260619 12:13:26.684208 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:26.684392 804 external_mini_cluster.cc:1428] Setting key 5c40a0ce0dada30ff0ab59c5903bf90c
I20260619 12:13:26.684568 21499 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d64369ae444744eaaebbcb85a422c619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64369ae444744eaaebbcb85a422c619" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39463 } } }
I20260619 12:13:26.684712 21499 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:26.684914 21507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:26.685482 21507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:26.688060 21507 catalog_manager.cc:1347] Generated new cluster ID: b965cb21f9594e8fbc963483730fc375
I20260619 12:13:26.688112 21507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:26.690331 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:26.685027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54323 (local address 127.0.0.1:39463)
0619 12:13:26.685241 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:26.685250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:26.685284 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:13:26.685398 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:26.685404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:26.685474 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:26.685838 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:26.685925 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:26.687629 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:26.688553 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:26.688980 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:26.689094 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:26.689742 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:26.689746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:26.689746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:26.689920 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:26.689949 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:26.689959 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:26.690035 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:26.690106 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 12:13:26.690165 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260619 12:13:26.691186 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:26.695421 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:26.692681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44243 (local address 127.0.0.1:39463)
0619 12:13:26.692715 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:26.692717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:26.692865 (+ 148us) server_negotiation.cc:407] Connection header received
0619 12:13:26.692932 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:26.692933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:26.692943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:26.692997 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:26.693114 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:26.693576 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:26.694258 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:26.694457 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:26.694531 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:26.695147 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:26.695148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:26.695148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:26.695152 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:26.695165 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:26.695168 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:26.695265 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:26.695325 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 12:13:26.695347 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:26.741811 21507 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:26.742580 21507 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:26.745357 21507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d64369ae444744eaaebbcb85a422c619: Generated new TSK 0
I20260619 12:13:26.745575 21507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:26.752885 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21427
2026-06-19T12:13:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1360 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:26 dist-test-slave-c910 krb5kdc[21526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:26 dist-test-slave-c910 krb5kdc[21526](info): set up 2 sockets
Jun 19 12:13:26 dist-test-slave-c910 krb5kdc[21526](info): commencing operation
krb5kdc: starting...
W20260619 12:13:27.795382 804 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:27 dist-test-slave-c910 krb5kdc[21526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:27.905925 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:27.984015 21539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:27.984242 21539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:27.984279 21539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:27.985416 21539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:27.985484 21539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:27.985512 21539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:27.985543 21539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:27.985567 21539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:27.986977 21539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:27.987264 21539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:27.987488 21539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:27.992084 21544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:27.992069 21545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:27.992062 21547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:27.992292 21539 server_base.cc:1034] running on GCE node
I20260619 12:13:27.992527 21539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:27.992810 21539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:27.993975 21539 hybrid_clock.cc:648] HybridClock initialized: now 1781871207993968 us; error 39 us; skew 500 ppm
Jun 19 12:13:27 dist-test-slave-c910 krb5kdc[21526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:27.996390 21539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:27.996838 21539 webserver.cc:460] Webserver started at http://127.0.0.1:41293/ using document root <none> and password file <none>
I20260619 12:13:27.997071 21539 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:27.997138 21539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:27.997283 21539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:27.998263 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "8d43031ff72544b083062b6aa298766c"
format_stamp: "Formatted at 2026-06-19 12:13:27 on dist-test-slave-c910"
server_key: "086bd254a0f406b2839f7adf84cd825b"
server_key_iv: "548a11383c82d6c7acdd0d3ef568d634"
server_key_version: "encryptionkey@0"
I20260619 12:13:27.998612 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "8d43031ff72544b083062b6aa298766c"
format_stamp: "Formatted at 2026-06-19 12:13:27 on dist-test-slave-c910"
server_key: "086bd254a0f406b2839f7adf84cd825b"
server_key_iv: "548a11383c82d6c7acdd0d3ef568d634"
server_key_version: "encryptionkey@0"
I20260619 12:13:27.999966 21539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:28.000609 21554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:28.000801 21539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:28.000901 21539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "8d43031ff72544b083062b6aa298766c"
format_stamp: "Formatted at 2026-06-19 12:13:27 on dist-test-slave-c910"
server_key: "086bd254a0f406b2839f7adf84cd825b"
server_key_iv: "548a11383c82d6c7acdd0d3ef568d634"
server_key_version: "encryptionkey@0"
I20260619 12:13:28.000979 21539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:28.004206 21539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:28.005927 21539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:28.006075 21539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:28.010053 21539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35189
I20260619 12:13:28.010102 21606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35189 every 8 connection(s)
I20260619 12:13:28.010418 21539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:28.011395 21607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:28.013561 21607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c: Bootstrap starting.
I20260619 12:13:28.014128 21607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:28.014411 21607 log.cc:824] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:28.015051 21607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c: No bootstrap required, opened a new log
I20260619 12:13:28.016651 21607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d43031ff72544b083062b6aa298766c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35189 } }
I20260619 12:13:28.016790 21607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:28.016837 21607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d43031ff72544b083062b6aa298766c, State: Initialized, Role: FOLLOWER
I20260619 12:13:28.016979 21607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d43031ff72544b083062b6aa298766c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35189 } }
I20260619 12:13:28.017053 21607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:28.017096 21607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:28.017148 21607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:28.017649 21607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d43031ff72544b083062b6aa298766c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35189 } }
I20260619 12:13:28.017763 21607 leader_election.cc:302] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d43031ff72544b083062b6aa298766c; no voters:
I20260619 12:13:28.017936 21607 leader_election.cc:288] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:28.017974 21610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:28.018131 21610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [term 1 LEADER]: Becoming Leader. State: Replica: 8d43031ff72544b083062b6aa298766c, State: Running, Role: LEADER
I20260619 12:13:28.018181 21607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:28.018280 21610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d43031ff72544b083062b6aa298766c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35189 } }
I20260619 12:13:28.018915 21612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d43031ff72544b083062b6aa298766c. Latest consensus state: current_term: 1 leader_uuid: "8d43031ff72544b083062b6aa298766c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d43031ff72544b083062b6aa298766c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35189 } } }
I20260619 12:13:28.019001 21612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:28.018922 21611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d43031ff72544b083062b6aa298766c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d43031ff72544b083062b6aa298766c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35189 } } }
I20260619 12:13:28.019232 21611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:28.019270 21619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:28.019793 21619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:28.019886 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21539
I20260619 12:13:28.019996 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:28.020191 804 external_mini_cluster.cc:1428] Setting key 2241f87e8ade2c98a9b550f5aee7a871
I20260619 12:13:28.021427 21619 catalog_manager.cc:1347] Generated new cluster ID: 00173ed876454c0b89bb1c5d9ab47f11
I20260619 12:13:28.021488 21619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:28.022531 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:28.020606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56267 (local address 127.0.0.1:35189)
0619 12:13:28.020746 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:28.020754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:28.021095 (+ 341us) server_negotiation.cc:407] Connection header received
0619 12:13:28.021211 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:28.021219 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:28.021259 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:28.021639 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:28.022100 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:28.022104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:28.022104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:28.022107 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:28.022137 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:28.022150 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:28.022205 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:28.022333 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 12:13:28.022372 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260619 12:13:28.023445 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:28.025990 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:28.025044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53045 (local address 127.0.0.1:35189)
0619 12:13:28.025092 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:28.025093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:28.025102 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:28.025122 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:28.025123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:28.025127 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:28.025189 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:28.025772 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:28.025772 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:28.025773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:28.025777 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:28.025791 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:28.025795 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:28.025857 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:28.025918 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 12:13:28.025930 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 12:13:28.037448 21619 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:28.037626 21619 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:28.045629 21619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d43031ff72544b083062b6aa298766c: Generated new TSK 0
I20260619 12:13:28.045806 21619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:28.052474 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21539
2026-06-19T12:13:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:28 dist-test-slave-c910 krb5kdc[21638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:28 dist-test-slave-c910 krb5kdc[21638](info): set up 2 sockets
Jun 19 12:13:28 dist-test-slave-c910 krb5kdc[21638](info): commencing operation
krb5kdc: starting...
W20260619 12:13:29.071406 804 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:29 dist-test-slave-c910 krb5kdc[21638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:29.185678 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:29.266692 21651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:29.266876 21651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:29.266897 21651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:29.268002 21651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:29.268042 21651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:29.268056 21651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:29.268069 21651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:29.268080 21651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:29.269433 21651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:29.269608 21651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:29.269773 21651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:29.274608 21656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:29.274713 21651 server_base.cc:1034] running on GCE node
W20260619 12:13:29.274598 21657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:29.274580 21659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:29.275072 21651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:29.275372 21651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:29.276516 21651 hybrid_clock.cc:648] HybridClock initialized: now 1781871209276504 us; error 30 us; skew 500 ppm
Jun 19 12:13:29 dist-test-slave-c910 krb5kdc[21638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:29.278889 21651 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:29.279347 21651 webserver.cc:460] Webserver started at http://127.0.0.1:46147/ using document root <none> and password file <none>
I20260619 12:13:29.279587 21651 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:29.279630 21651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:29.279790 21651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:29.280800 21651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "6de79af9d9e14d1cb13c8b411d586d87"
format_stamp: "Formatted at 2026-06-19 12:13:29 on dist-test-slave-c910"
server_key: "bb588d061d5bdab1d4a1660fb051fc1a"
server_key_iv: "e429ac432c173c55b9b4aadf3af30d33"
server_key_version: "encryptionkey@0"
I20260619 12:13:29.281139 21651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "6de79af9d9e14d1cb13c8b411d586d87"
format_stamp: "Formatted at 2026-06-19 12:13:29 on dist-test-slave-c910"
server_key: "bb588d061d5bdab1d4a1660fb051fc1a"
server_key_iv: "e429ac432c173c55b9b4aadf3af30d33"
server_key_version: "encryptionkey@0"
I20260619 12:13:29.282433 21651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:29.283121 21666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:29.283331 21651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:29.283427 21651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "6de79af9d9e14d1cb13c8b411d586d87"
format_stamp: "Formatted at 2026-06-19 12:13:29 on dist-test-slave-c910"
server_key: "bb588d061d5bdab1d4a1660fb051fc1a"
server_key_iv: "e429ac432c173c55b9b4aadf3af30d33"
server_key_version: "encryptionkey@0"
I20260619 12:13:29.283512 21651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:29.294880 21651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:29.297083 21651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:29.297245 21651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:29.301170 21651 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40745
I20260619 12:13:29.301224 21718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40745 every 8 connection(s)
I20260619 12:13:29.301563 21651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:29.302402 21719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:29.304687 21719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87: Bootstrap starting.
I20260619 12:13:29.305246 21719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:29.305531 21719 log.cc:824] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:29.306177 21719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87: No bootstrap required, opened a new log
I20260619 12:13:29.307897 21719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de79af9d9e14d1cb13c8b411d586d87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 12:13:29.308025 21719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:29.308081 21719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de79af9d9e14d1cb13c8b411d586d87, State: Initialized, Role: FOLLOWER
I20260619 12:13:29.308225 21719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de79af9d9e14d1cb13c8b411d586d87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 12:13:29.308284 21719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:29.308323 21719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:29.308389 21719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:29.308945 21719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de79af9d9e14d1cb13c8b411d586d87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 12:13:29.309103 21719 leader_election.cc:302] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6de79af9d9e14d1cb13c8b411d586d87; no voters:
I20260619 12:13:29.309311 21719 leader_election.cc:288] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:29.309326 21722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:29.309513 21722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [term 1 LEADER]: Becoming Leader. State: Replica: 6de79af9d9e14d1cb13c8b411d586d87, State: Running, Role: LEADER
I20260619 12:13:29.309579 21719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:29.309652 21722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de79af9d9e14d1cb13c8b411d586d87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 12:13:29.309930 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21651
I20260619 12:13:29.310078 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:29.310053 21724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6de79af9d9e14d1cb13c8b411d586d87. Latest consensus state: current_term: 1 leader_uuid: "6de79af9d9e14d1cb13c8b411d586d87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de79af9d9e14d1cb13c8b411d586d87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260619 12:13:29.310122 21724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:29.310261 21723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6de79af9d9e14d1cb13c8b411d586d87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de79af9d9e14d1cb13c8b411d586d87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260619 12:13:29.310307 21723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:29.310316 804 external_mini_cluster.cc:1428] Setting key 9172a72c3771f09bfe8b4c259a7bd630
I20260619 12:13:29.310540 21726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:29.311079 21726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:29.312707 21726 catalog_manager.cc:1347] Generated new cluster ID: cd35d4dc5273490c8b48186464b8933b
I20260619 12:13:29.312749 21726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 12:13:29 dist-test-slave-c910 krb5kdc[21638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:13:29.321945 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:29.310942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:37025 (local address 127.0.0.1:40745)
0619 12:13:29.311098 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:29.311107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:29.311137 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:13:29.311280 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:29.311288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:29.311361 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:29.311572 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:29.312394 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:29.313728 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:29.314816 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:29.315422 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:29.315516 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:29.318283 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:29.318288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:29.318289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:29.318304 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:29.320261 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:29.320694 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:29.320697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:29.320697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:29.320830 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:29.321064 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:29.321064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:29.321065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:29.321370 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:29.321530 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:29.321616 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 12:13:29.321732 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260619 12:13:29.323108 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:29.326565 21726 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:29.327297 21726 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:29.330590 21726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6de79af9d9e14d1cb13c8b411d586d87: Generated new TSK 0
I20260619 12:13:29.330768 21726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:29.331034 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:29.324450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:43487 (local address 127.0.0.1:40745)
0619 12:13:29.324482 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:29.324484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:29.324729 (+ 245us) server_negotiation.cc:407] Connection header received
0619 12:13:29.324756 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:29.324757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:29.324768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:29.324828 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:29.325377 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:29.326157 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:29.326918 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:29.327135 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:29.327248 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:29.328381 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:29.328382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:29.328382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:29.328388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:29.329741 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:29.330110 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:29.330111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:29.330111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:29.330163 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:29.330428 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:29.330429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:29.330429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:29.330546 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:29.330635 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:29.330884 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 12:13:29.330945 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 12:13:29.332898 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21651
W20260619 12:13:29.336592 21636 connection.cc:535] client connection to 127.0.0.1:40745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40745): Connection reset by peer (error 104)
2026-06-19T12:13:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:29 dist-test-slave-c910 krb5kdc[21750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:29 dist-test-slave-c910 krb5kdc[21750](info): set up 2 sockets
Jun 19 12:13:29 dist-test-slave-c910 krb5kdc[21750](info): commencing operation
krb5kdc: starting...
W20260619 12:13:30.363420 804 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:30 dist-test-slave-c910 krb5kdc[21750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:30.476030 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:30.554924 21763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:30.555151 21763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:30.555188 21763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:30.556344 21763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:30.556414 21763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:30.556438 21763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:30.556468 21763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:30.556499 21763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:30.557940 21763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:30.558202 21763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:30.558445 21763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:30.563159 21768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:30.563234 21769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:30.563244 21771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:30.563274 21763 server_base.cc:1034] running on GCE node
I20260619 12:13:30.563596 21763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:30.563838 21763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:30.564976 21763 hybrid_clock.cc:648] HybridClock initialized: now 1781871210564958 us; error 47 us; skew 500 ppm
Jun 19 12:13:30 dist-test-slave-c910 krb5kdc[21750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871210, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:30.567407 21763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:30.567906 21763 webserver.cc:460] Webserver started at http://127.0.0.1:40387/ using document root <none> and password file <none>
I20260619 12:13:30.568136 21763 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:30.568181 21763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:30.568365 21763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:30.569339 21763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0"
format_stamp: "Formatted at 2026-06-19 12:13:30 on dist-test-slave-c910"
server_key: "8f815cbdc33c996cd5941ae425fd1081"
server_key_iv: "5e9ebf05aebb8ef66c903539889cd50e"
server_key_version: "encryptionkey@0"
I20260619 12:13:30.569700 21763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0"
format_stamp: "Formatted at 2026-06-19 12:13:30 on dist-test-slave-c910"
server_key: "8f815cbdc33c996cd5941ae425fd1081"
server_key_iv: "5e9ebf05aebb8ef66c903539889cd50e"
server_key_version: "encryptionkey@0"
I20260619 12:13:30.570989 21763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:30.571754 21778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:30.571951 21763 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:30.572031 21763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0"
format_stamp: "Formatted at 2026-06-19 12:13:30 on dist-test-slave-c910"
server_key: "8f815cbdc33c996cd5941ae425fd1081"
server_key_iv: "5e9ebf05aebb8ef66c903539889cd50e"
server_key_version: "encryptionkey@0"
I20260619 12:13:30.572080 21763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:30.583709 21763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:30.585943 21763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:30.586054 21763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:30.589814 21763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40173
I20260619 12:13:30.589900 21830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40173 every 8 connection(s)
I20260619 12:13:30.590220 21763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:30.591078 21831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:30.593308 21831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0: Bootstrap starting.
I20260619 12:13:30.593875 21831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:30.594162 21831 log.cc:824] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:30.594834 21831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0: No bootstrap required, opened a new log
I20260619 12:13:30.596441 21831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40173 } }
I20260619 12:13:30.596593 21831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:30.596637 21831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7336b1f1b2584e5eb6e9834fd05d7fc0, State: Initialized, Role: FOLLOWER
I20260619 12:13:30.596820 21831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40173 } }
I20260619 12:13:30.596920 21831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:30.596962 21831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:30.597016 21831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:30.597509 21831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40173 } }
I20260619 12:13:30.597625 21831 leader_election.cc:302] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7336b1f1b2584e5eb6e9834fd05d7fc0; no voters:
I20260619 12:13:30.597807 21831 leader_election.cc:288] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:30.597848 21834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:30.598016 21834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [term 1 LEADER]: Becoming Leader. State: Replica: 7336b1f1b2584e5eb6e9834fd05d7fc0, State: Running, Role: LEADER
I20260619 12:13:30.598060 21831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:30.598136 21834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40173 } }
I20260619 12:13:30.598740 21835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40173 } } }
I20260619 12:13:30.598762 21836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7336b1f1b2584e5eb6e9834fd05d7fc0. Latest consensus state: current_term: 1 leader_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7336b1f1b2584e5eb6e9834fd05d7fc0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40173 } } }
I20260619 12:13:30.598832 21835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:30.598855 21836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:30.599031 21843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:30.599623 21843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:30.600346 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21763
I20260619 12:13:30.600455 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:30.600664 804 external_mini_cluster.cc:1428] Setting key a5ab7697e916b346ffbe30ce0fd73aab
I20260619 12:13:30.601789 21843 catalog_manager.cc:1347] Generated new cluster ID: 01ac228628ac403882c03dc7710bb5c3
I20260619 12:13:30.601850 21843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:30.607695 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:30.601135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:53843 (local address 127.0.0.1:40173)
0619 12:13:30.601873 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:30.601881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:30.601911 (+ 30us) server_negotiation.cc:407] Connection header received
0619 12:13:30.602034 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:30.602041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:30.602099 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:30.602481 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:30.602570 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:30.604046 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:30.605203 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:30.605833 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:30.606262 (+ 429us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:30.606719 (+ 457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:30.606722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:30.606723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:30.606927 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:30.606963 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:30.606978 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:30.607319 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:30.607373 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:13:30.607409 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":699,"thread_start_us":86,"threads_started":1}
I20260619 12:13:30.608554 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:30.615020 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:30.611450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:55187 (local address 127.0.0.1:40173)
0619 12:13:30.611487 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:30.611488 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:30.611717 (+ 229us) server_negotiation.cc:407] Connection header received
0619 12:13:30.611746 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:30.611747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:30.611757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:30.611817 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:30.611945 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:30.612700 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:30.613675 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:30.614003 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:30.614071 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:30.614769 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:30.614770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:30.614770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:30.614775 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:30.614790 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:30.614794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:30.614857 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:30.614933 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 12:13:30.614944 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:30.615837 21843 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:30.616290 21843 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:30.621707 21843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7336b1f1b2584e5eb6e9834fd05d7fc0: Generated new TSK 0
I20260619 12:13:30.621903 21843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:30.624302 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21763
W20260619 12:13:30.628129 21748 connection.cc:535] client connection to 127.0.0.1:40173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40173): Connection reset by peer (error 104)
2026-06-19T12:13:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:30 dist-test-slave-c910 krb5kdc[21862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:30 dist-test-slave-c910 krb5kdc[21862](info): set up 2 sockets
Jun 19 12:13:30 dist-test-slave-c910 krb5kdc[21862](info): commencing operation
krb5kdc: starting...
W20260619 12:13:31.647341 804 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:31 dist-test-slave-c910 krb5kdc[21862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:31.769428 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:31.849332 21875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:31.849525 21875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:31.849548 21875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:31.850698 21875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:31.850742 21875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:31.850756 21875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:31.850768 21875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:31.850821 21875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:31.852294 21875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:31.852571 21875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:31.852802 21875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:31.857813 21875 server_base.cc:1034] running on GCE node
W20260619 12:13:31.857776 21880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:31.857744 21883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:31.857759 21881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:31.858186 21875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:31.858469 21875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:31.859606 21875 hybrid_clock.cc:648] HybridClock initialized: now 1781871211859595 us; error 29 us; skew 500 ppm
Jun 19 12:13:31 dist-test-slave-c910 krb5kdc[21862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871211, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:31.866686 21875 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:31.867110 21875 webserver.cc:460] Webserver started at http://127.0.0.1:40917/ using document root <none> and password file <none>
I20260619 12:13:31.867394 21875 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:31.867466 21875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:31.867615 21875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:31.868655 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "51cf8a0275d541cf9c73954c1d898431"
format_stamp: "Formatted at 2026-06-19 12:13:31 on dist-test-slave-c910"
server_key: "6338f7fab68b79eed382dff64d72239f"
server_key_iv: "56435c8e826502ee06110aa37e4c3349"
server_key_version: "encryptionkey@0"
I20260619 12:13:31.869073 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "51cf8a0275d541cf9c73954c1d898431"
format_stamp: "Formatted at 2026-06-19 12:13:31 on dist-test-slave-c910"
server_key: "6338f7fab68b79eed382dff64d72239f"
server_key_iv: "56435c8e826502ee06110aa37e4c3349"
server_key_version: "encryptionkey@0"
I20260619 12:13:31.870375 21875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:31.871095 21890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:31.871317 21875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:31.871413 21875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "51cf8a0275d541cf9c73954c1d898431"
format_stamp: "Formatted at 2026-06-19 12:13:31 on dist-test-slave-c910"
server_key: "6338f7fab68b79eed382dff64d72239f"
server_key_iv: "56435c8e826502ee06110aa37e4c3349"
server_key_version: "encryptionkey@0"
I20260619 12:13:31.871484 21875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:31.874673 21875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:31.876530 21875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:31.876686 21875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:31.880546 21875 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41865
I20260619 12:13:31.880575 21942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41865 every 8 connection(s)
I20260619 12:13:31.880896 21875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:31.881867 21943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:31.883967 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21875
I20260619 12:13:31.884099 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:31.884076 21943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431: Bootstrap starting.
I20260619 12:13:31.884274 804 external_mini_cluster.cc:1428] Setting key 4912ddd09ca153c4f9a8f5dc675809b5
I20260619 12:13:31.884724 21943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:31.885037 21943 log.cc:824] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:31.885749 21943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431: No bootstrap required, opened a new log
I20260619 12:13:31.886617 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:31.884745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:36915 (local address 127.0.0.1:41865)
0619 12:13:31.884944 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:31.884951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:31.884975 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:13:31.885429 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:31.885436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:31.885462 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:31.885573 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:31.886199 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:31.886202 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:31.886202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:31.886206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:31.886239 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:31.886249 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:31.886318 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:31.886384 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 12:13:31.886437 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260619 12:13:31.887650 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:31.888170 21943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51cf8a0275d541cf9c73954c1d898431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41865 } }
I20260619 12:13:31.888302 21943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:31.888325 21943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51cf8a0275d541cf9c73954c1d898431, State: Initialized, Role: FOLLOWER
I20260619 12:13:31.888445 21943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51cf8a0275d541cf9c73954c1d898431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41865 } }
I20260619 12:13:31.888530 21943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:31.888552 21943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:31.888616 21943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:31.889294 21943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51cf8a0275d541cf9c73954c1d898431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41865 } }
I20260619 12:13:31.889405 21943 leader_election.cc:302] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51cf8a0275d541cf9c73954c1d898431; no voters:
I20260619 12:13:31.889560 21943 leader_election.cc:288] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:31.889611 21953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:31.889780 21953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [term 1 LEADER]: Becoming Leader. State: Replica: 51cf8a0275d541cf9c73954c1d898431, State: Running, Role: LEADER
I20260619 12:13:31.889801 21943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:31.890204 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:31.889081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:56083 (local address 127.0.0.1:41865)
0619 12:13:31.889117 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:31.889118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:31.889174 (+ 56us) server_negotiation.cc:407] Connection header received
0619 12:13:31.889247 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:31.889248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:31.889253 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:31.889305 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:31.889908 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:31.889909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:31.889909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:31.889913 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:31.889925 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:31.889929 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:31.889978 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:31.890128 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 12:13:31.890143 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:13:31.890535 21953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51cf8a0275d541cf9c73954c1d898431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41865 } }
I20260619 12:13:31.891067 21955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51cf8a0275d541cf9c73954c1d898431. Latest consensus state: current_term: 1 leader_uuid: "51cf8a0275d541cf9c73954c1d898431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51cf8a0275d541cf9c73954c1d898431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41865 } } }
I20260619 12:13:31.891299 21954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51cf8a0275d541cf9c73954c1d898431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51cf8a0275d541cf9c73954c1d898431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41865 } } }
I20260619 12:13:31.891161 21955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:31.891498 21954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431 [sys.catalog]: This master's current role is: LEADER
W20260619 12:13:31.891685 21967 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:31.891741 21967 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:31.891906 21969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:31.892127 21969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:31.893537 21969 catalog_manager.cc:1347] Generated new cluster ID: 073341b9711c47058244c6da66071fc9
I20260619 12:13:31.893576 21969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:31.904958 21969 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:31.905143 21969 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:31.907919 21969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51cf8a0275d541cf9c73954c1d898431: Generated new TSK 0
I20260619 12:13:31.908097 21969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:31.911854 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21875
2026-06-19T12:13:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:31 dist-test-slave-c910 krb5kdc[21974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:31 dist-test-slave-c910 krb5kdc[21974](info): set up 2 sockets
Jun 19 12:13:31 dist-test-slave-c910 krb5kdc[21974](info): commencing operation
krb5kdc: starting...
W20260619 12:13:32.939291 804 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:33 dist-test-slave-c910 krb5kdc[21974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:33.055523 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:33.136353 21987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:33.136580 21987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:33.136617 21987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:33.137759 21987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:33.137823 21987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:33.137848 21987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:33.137876 21987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:33.137907 21987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:33.139359 21987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:33.139621 21987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:33.139851 21987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:33.144639 21993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:33.144614 21995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:33.144639 21992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:33.144726 21987 server_base.cc:1034] running on GCE node
I20260619 12:13:33.145040 21987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:33.145331 21987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:33.146466 21987 hybrid_clock.cc:648] HybridClock initialized: now 1781871213146445 us; error 34 us; skew 500 ppm
Jun 19 12:13:33 dist-test-slave-c910 krb5kdc[21974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:33.148864 21987 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:33.149272 21987 webserver.cc:460] Webserver started at http://127.0.0.1:45447/ using document root <none> and password file <none>
I20260619 12:13:33.149559 21987 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:33.149629 21987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:33.149775 21987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:33.150790 21987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "c95eeeacec344c59be56e44a46512266"
format_stamp: "Formatted at 2026-06-19 12:13:33 on dist-test-slave-c910"
server_key: "6e60c277b5b7336f18647dfc6d8cad0d"
server_key_iv: "17df4e67571c367ff1702547896448c4"
server_key_version: "encryptionkey@0"
I20260619 12:13:33.151140 21987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "c95eeeacec344c59be56e44a46512266"
format_stamp: "Formatted at 2026-06-19 12:13:33 on dist-test-slave-c910"
server_key: "6e60c277b5b7336f18647dfc6d8cad0d"
server_key_iv: "17df4e67571c367ff1702547896448c4"
server_key_version: "encryptionkey@0"
I20260619 12:13:33.152433 21987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:33.153175 22002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:33.153338 21987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:33.153440 21987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "c95eeeacec344c59be56e44a46512266"
format_stamp: "Formatted at 2026-06-19 12:13:33 on dist-test-slave-c910"
server_key: "6e60c277b5b7336f18647dfc6d8cad0d"
server_key_iv: "17df4e67571c367ff1702547896448c4"
server_key_version: "encryptionkey@0"
I20260619 12:13:33.153512 21987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:33.169883 21987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:33.171720 21987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:33.171878 21987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:33.175647 21987 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35963
I20260619 12:13:33.175818 22054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35963 every 8 connection(s)
I20260619 12:13:33.176081 21987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:33.176952 22055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:33.179111 22055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266: Bootstrap starting.
I20260619 12:13:33.179757 22055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:33.179893 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 21987
I20260619 12:13:33.180027 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:33.180104 22055 log.cc:824] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:33.180245 804 external_mini_cluster.cc:1428] Setting key 444ae85d9f9d1945324e57d647a68727
I20260619 12:13:33.180971 22055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266: No bootstrap required, opened a new log
I20260619 12:13:33.183607 22055 raft_consensus.cc:348] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95eeeacec344c59be56e44a46512266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35963 } }
I20260619 12:13:33.183782 22055 raft_consensus.cc:374] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:33.183820 22055 raft_consensus.cc:729] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95eeeacec344c59be56e44a46512266, State: Initialized, Role: FOLLOWER
I20260619 12:13:33.183943 22055 consensus_queue.cc:260] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95eeeacec344c59be56e44a46512266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35963 } }
I20260619 12:13:33.184018 22055 raft_consensus.cc:388] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:33.184046 22055 raft_consensus.cc:482] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:33.184088 22055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:33.184703 22055 raft_consensus.cc:504] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95eeeacec344c59be56e44a46512266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35963 } }
I20260619 12:13:33.184792 22055 leader_election.cc:302] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c95eeeacec344c59be56e44a46512266; no voters:
I20260619 12:13:33.184995 22055 leader_election.cc:288] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:33.185045 22060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:33.185214 22060 raft_consensus.cc:686] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [term 1 LEADER]: Becoming Leader. State: Replica: c95eeeacec344c59be56e44a46512266, State: Running, Role: LEADER
I20260619 12:13:33.185362 22060 consensus_queue.cc:237] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95eeeacec344c59be56e44a46512266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35963 } }
I20260619 12:13:33.185653 22055 sys_catalog.cc:564] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:33.185698 22061 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c95eeeacec344c59be56e44a46512266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95eeeacec344c59be56e44a46512266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35963 } } }
I20260619 12:13:33.185781 22061 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:33.185950 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c95eeeacec344c59be56e44a46512266. Latest consensus state: current_term: 1 leader_uuid: "c95eeeacec344c59be56e44a46512266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95eeeacec344c59be56e44a46512266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35963 } } }
I20260619 12:13:33.186085 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266 [sys.catalog]: This master's current role is: LEADER
Jun 19 12:13:33 dist-test-slave-c910 krb5kdc[21974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260619 12:13:33.187530 22076 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:33.187594 22076 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:33.187695 22064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:33.187870 22064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:33.189417 22064 catalog_manager.cc:1347] Generated new cluster ID: 8ad46272feb04a1c874888350854a77b
I20260619 12:13:33.189478 22064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:33.191537 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:33.180830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:35099 (local address 127.0.0.1:35963)
0619 12:13:33.180985 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:33.180994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:33.181027 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:13:33.181484 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:33.181490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:33.181539 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:33.181647 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:33.182433 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:33.183357 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:33.184419 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:33.184849 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:33.184975 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:33.187743 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:33.187746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:33.187747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:33.187757 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:33.189695 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:33.190115 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:33.190117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:33.190118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:33.190224 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:33.190476 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:33.190477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:33.190477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:33.190771 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:33.190951 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:33.191049 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:13:33.191159 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260619 12:13:33.192464 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:33.198577 22064 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:33.199225 22064 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:33.201148 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:33.194003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52333 (local address 127.0.0.1:35963)
0619 12:13:33.194041 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:33.194042 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:33.194180 (+ 138us) server_negotiation.cc:407] Connection header received
0619 12:13:33.194209 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:33.194209 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:33.194219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:33.194271 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:33.194861 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:33.195612 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:33.196601 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:33.196914 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:33.196973 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:33.198375 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:33.198376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:33.198377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:33.198383 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:33.199973 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:33.200377 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:33.200378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:33.200379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:33.200435 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:33.200652 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:33.200653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:33.200653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:33.200772 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:33.200871 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:33.200979 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 12:13:33.201039 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:13:33.204783 22064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c95eeeacec344c59be56e44a46512266: Generated new TSK 0
I20260619 12:13:33.204943 22064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:33.211262 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 21987
W20260619 12:13:33.215391 21972 connection.cc:535] client connection to 127.0.0.1:35963 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35963): Connection reset by peer (error 104)
2026-06-19T12:13:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:33 dist-test-slave-c910 krb5kdc[22086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:33 dist-test-slave-c910 krb5kdc[22086](info): set up 2 sockets
Jun 19 12:13:33 dist-test-slave-c910 krb5kdc[22086](info): commencing operation
krb5kdc: starting...
W20260619 12:13:34.335386 804 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:34 dist-test-slave-c910 krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:34.454344 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:34.535753 22099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:34.536010 22099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:34.536072 22099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:34.537263 22099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:34.537335 22099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:34.537360 22099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:34.537390 22099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:34.537415 22099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:34.538920 22099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:34.539269 22099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:34.539512 22099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:34.544523 22099 server_base.cc:1034] running on GCE node
W20260619 12:13:34.544555 22104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:34.544641 22107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:34.544721 22105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:34.544994 22099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:34.545284 22099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:34.546475 22099 hybrid_clock.cc:648] HybridClock initialized: now 1781871214546453 us; error 36 us; skew 500 ppm
Jun 19 12:13:34 dist-test-slave-c910 krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:34.549033 22099 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:34.549440 22099 webserver.cc:460] Webserver started at http://127.0.0.1:45373/ using document root <none> and password file <none>
I20260619 12:13:34.549678 22099 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:34.549721 22099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:34.549851 22099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:34.550920 22099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "54f48cc8f13a452ca87e16a60a976186"
format_stamp: "Formatted at 2026-06-19 12:13:34 on dist-test-slave-c910"
server_key: "54fdf142f98d996b01c51ae2d22bdfa7"
server_key_iv: "e848a544a98ba3636bb8e63ef6db32e1"
server_key_version: "encryptionkey@0"
I20260619 12:13:34.551338 22099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "54f48cc8f13a452ca87e16a60a976186"
format_stamp: "Formatted at 2026-06-19 12:13:34 on dist-test-slave-c910"
server_key: "54fdf142f98d996b01c51ae2d22bdfa7"
server_key_iv: "e848a544a98ba3636bb8e63ef6db32e1"
server_key_version: "encryptionkey@0"
I20260619 12:13:34.552683 22099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:34.553435 22114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:34.553609 22099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:34.553706 22099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "54f48cc8f13a452ca87e16a60a976186"
format_stamp: "Formatted at 2026-06-19 12:13:34 on dist-test-slave-c910"
server_key: "54fdf142f98d996b01c51ae2d22bdfa7"
server_key_iv: "e848a544a98ba3636bb8e63ef6db32e1"
server_key_version: "encryptionkey@0"
I20260619 12:13:34.553782 22099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:34.598889 22099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:34.601009 22099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:34.601177 22099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:34.605065 22099 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37159
I20260619 12:13:34.605129 22166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37159 every 8 connection(s)
I20260619 12:13:34.605479 22099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:34.606454 22167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:34.608897 22167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186: Bootstrap starting.
I20260619 12:13:34.609398 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22099
I20260619 12:13:34.609514 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:34.609622 22167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:34.609733 804 external_mini_cluster.cc:1428] Setting key 7ed7db68d3a7b3412bef30c8f801f58d
I20260619 12:13:34.609956 22167 log.cc:824] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:34.610898 22167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186: No bootstrap required, opened a new log
I20260619 12:13:34.612401 22167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f48cc8f13a452ca87e16a60a976186" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260619 12:13:34.612587 22167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:34.612627 22167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f48cc8f13a452ca87e16a60a976186, State: Initialized, Role: FOLLOWER
I20260619 12:13:34.612766 22167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f48cc8f13a452ca87e16a60a976186" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260619 12:13:34.612841 22167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:34.612871 22167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:34.612916 22167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:34.613569 22167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f48cc8f13a452ca87e16a60a976186" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260619 12:13:34.613687 22167 leader_election.cc:302] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54f48cc8f13a452ca87e16a60a976186; no voters:
I20260619 12:13:34.613863 22167 leader_election.cc:288] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:34.613893 22172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:34.614003 22172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [term 1 LEADER]: Becoming Leader. State: Replica: 54f48cc8f13a452ca87e16a60a976186, State: Running, Role: LEADER
I20260619 12:13:34.614126 22167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:34.614154 22172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f48cc8f13a452ca87e16a60a976186" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } }
I20260619 12:13:34.614526 22172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54f48cc8f13a452ca87e16a60a976186. Latest consensus state: current_term: 1 leader_uuid: "54f48cc8f13a452ca87e16a60a976186" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f48cc8f13a452ca87e16a60a976186" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } } }
I20260619 12:13:34.614537 22173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54f48cc8f13a452ca87e16a60a976186" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f48cc8f13a452ca87e16a60a976186" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37159 } } }
I20260619 12:13:34.614694 22172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:34.615085 22173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:34.615414 22181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:34.615931 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:34.610270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:41607 (local address 127.0.0.1:37159)
0619 12:13:34.610440 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:34.610448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:34.610475 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:13:34.610890 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:34.610896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:34.610972 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:34.611106 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:34.611338 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:34.612180 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:34.613518 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:34.614074 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:34.614206 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:34.614945 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:34.614949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:34.614950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:34.615174 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:34.615242 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:34.615268 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:34.615641 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:34.615692 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 12:13:34.615724 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260619 12:13:34.615969 22181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:34.616899 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:34.617481 22181 catalog_manager.cc:1347] Generated new cluster ID: 8f4c5d2197df4f6f9fabdfab58d03ca6
I20260619 12:13:34.617529 22181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:34.621800 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:34.618364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:58245 (local address 127.0.0.1:37159)
0619 12:13:34.618395 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:34.618396 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:34.618472 (+ 76us) server_negotiation.cc:407] Connection header received
0619 12:13:34.618537 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:34.618538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:34.618548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:34.618605 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:34.618726 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:34.619443 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:34.620458 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:34.620773 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:34.620840 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:34.621538 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:34.621539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:34.621540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:34.621544 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:34.621557 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:34.621561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:34.621609 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:34.621712 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:13:34.621722 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 12:13:34.627362 22181 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:34.627890 22181 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:34.636137 22181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54f48cc8f13a452ca87e16a60a976186: Generated new TSK 0
I20260619 12:13:34.636337 22181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:34.640872 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22099
W20260619 12:13:34.644470 22084 connection.cc:535] client connection to 127.0.0.1:37159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37159): Connection reset by peer (error 104)
2026-06-19T12:13:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:34 dist-test-slave-c910 krb5kdc[22198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:34 dist-test-slave-c910 krb5kdc[22198](info): set up 2 sockets
Jun 19 12:13:34 dist-test-slave-c910 krb5kdc[22198](info): commencing operation
krb5kdc: starting...
W20260619 12:13:35.679404 804 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:35 dist-test-slave-c910 krb5kdc[22198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:13:35.797982 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:35.877738 22211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:35.877969 22211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:35.878007 22211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:35.879141 22211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:35.879209 22211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:35.879259 22211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:35.879300 22211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:35.879323 22211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:35.880738 22211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:35.881006 22211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:35.881223 22211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:35.885879 22217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:35.885865 22216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:35.885895 22219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:35.886180 22211 server_base.cc:1034] running on GCE node
I20260619 12:13:35.886394 22211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:35.886613 22211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:35.887754 22211 hybrid_clock.cc:648] HybridClock initialized: now 1781871215887741 us; error 29 us; skew 500 ppm
Jun 19 12:13:35 dist-test-slave-c910 krb5kdc[22198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:35.890096 22211 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:13:35.890457 22211 webserver.cc:460] Webserver started at http://127.0.0.1:43093/ using document root <none> and password file <none>
I20260619 12:13:35.890627 22211 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:35.890669 22211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:35.890754 22211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:35.891779 22211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "648c4d479cb84320afb033216f2c71ff"
format_stamp: "Formatted at 2026-06-19 12:13:35 on dist-test-slave-c910"
server_key: "acba980d5a7aeb61eb090e525b45a9df"
server_key_iv: "ee2b96cb3092c88223c9cda7d41e52ba"
server_key_version: "encryptionkey@0"
I20260619 12:13:35.892068 22211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "648c4d479cb84320afb033216f2c71ff"
format_stamp: "Formatted at 2026-06-19 12:13:35 on dist-test-slave-c910"
server_key: "acba980d5a7aeb61eb090e525b45a9df"
server_key_iv: "ee2b96cb3092c88223c9cda7d41e52ba"
server_key_version: "encryptionkey@0"
I20260619 12:13:35.893364 22211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:13:35.894026 22226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:35.894194 22211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:35.894294 22211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "648c4d479cb84320afb033216f2c71ff"
format_stamp: "Formatted at 2026-06-19 12:13:35 on dist-test-slave-c910"
server_key: "acba980d5a7aeb61eb090e525b45a9df"
server_key_iv: "ee2b96cb3092c88223c9cda7d41e52ba"
server_key_version: "encryptionkey@0"
I20260619 12:13:35.894371 22211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:35.897576 22211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:35.899376 22211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:35.899528 22211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:35.903393 22211 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43419
I20260619 12:13:35.903465 22278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43419 every 8 connection(s)
I20260619 12:13:35.903769 22211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:35.904692 22279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:35.906924 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff: Bootstrap starting.
I20260619 12:13:35.907560 22279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:35.907845 22279 log.cc:824] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:35.908509 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff: No bootstrap required, opened a new log
I20260619 12:13:35.910099 22279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c4d479cb84320afb033216f2c71ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 12:13:35.910233 22279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:35.910257 22279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648c4d479cb84320afb033216f2c71ff, State: Initialized, Role: FOLLOWER
I20260619 12:13:35.910413 22279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c4d479cb84320afb033216f2c71ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 12:13:35.910475 22279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:35.910496 22279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:35.910522 22279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:35.911049 22279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c4d479cb84320afb033216f2c71ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 12:13:35.911159 22279 leader_election.cc:302] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 648c4d479cb84320afb033216f2c71ff; no voters:
I20260619 12:13:35.911338 22279 leader_election.cc:288] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:35.911401 22282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:35.911548 22282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [term 1 LEADER]: Becoming Leader. State: Replica: 648c4d479cb84320afb033216f2c71ff, State: Running, Role: LEADER
I20260619 12:13:35.911612 22279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:35.911702 22282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c4d479cb84320afb033216f2c71ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 12:13:35.912281 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648c4d479cb84320afb033216f2c71ff. Latest consensus state: current_term: 1 leader_uuid: "648c4d479cb84320afb033216f2c71ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c4d479cb84320afb033216f2c71ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } } }
I20260619 12:13:35.912405 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:35.912268 22283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "648c4d479cb84320afb033216f2c71ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648c4d479cb84320afb033216f2c71ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } } }
I20260619 12:13:35.912673 22283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:35.912591 22291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:35.912840 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22211
I20260619 12:13:35.912969 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:35.913172 804 external_mini_cluster.cc:1428] Setting key 8690b2277050c14bc1232478716f83f5
I20260619 12:13:35.913410 22291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:35.914968 22291 catalog_manager.cc:1347] Generated new cluster ID: e20e1ca3c6194dccb679a543f58573b1
I20260619 12:13:35.915021 22291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:35.920058 22301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:35.913657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:55471 (local address 127.0.0.1:43419)
0619 12:13:35.913814 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:35.913826 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:35.913929 (+ 103us) server_negotiation.cc:407] Connection header received
0619 12:13:35.914105 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:35.914112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:35.914152 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:35.914287 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:35.919554 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:35.919558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:35.919558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:35.919564 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:35.919614 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:35.919631 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:35.919727 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:35.919812 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 12:13:35.919861 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260619 12:13:35.921314 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:35.928455 22301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:35.922826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52107 (local address 127.0.0.1:43419)
0619 12:13:35.922871 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:35.922873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:35.927382 (+ 4509us) server_negotiation.cc:407] Connection header received
0619 12:13:35.927466 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:35.927467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:35.927492 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:35.927557 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:35.928223 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:35.928224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:35.928225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:35.928229 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:35.928244 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:35.928249 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:35.928300 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:35.928365 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 12:13:35.928382 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:13:35.931510 22291 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:35.931681 22291 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:35.936337 22291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 648c4d479cb84320afb033216f2c71ff: Generated new TSK 0
I20260619 12:13:35.936524 22291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:35.939282 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22211
2026-06-19T12:13:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:35 dist-test-slave-c910 krb5kdc[22310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:35 dist-test-slave-c910 krb5kdc[22310](info): set up 2 sockets
Jun 19 12:13:35 dist-test-slave-c910 krb5kdc[22310](info): commencing operation
krb5kdc: starting...
W20260619 12:13:36.983368 804 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:37 dist-test-slave-c910 krb5kdc[22310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:37.101370 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36707
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:36707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:37.180845 22323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:37.181059 22323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:37.181082 22323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:37.182142 22323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:37.182174 22323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:37.182188 22323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:37.182200 22323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:37.182212 22323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:37.183586 22323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:36707
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:37.183777 22323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:37.183940 22323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:37.188606 22329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:37.188598 22331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:37.188598 22328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:37.188905 22323 server_base.cc:1034] running on GCE node
I20260619 12:13:37.189132 22323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:37.189404 22323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:37.190568 22323 hybrid_clock.cc:648] HybridClock initialized: now 1781871217190553 us; error 28 us; skew 500 ppm
Jun 19 12:13:37 dist-test-slave-c910 krb5kdc[22310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871217, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:37.199118 22323 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:37.199563 22323 webserver.cc:460] Webserver started at http://127.0.201.62:43841/ using document root <none> and password file <none>
I20260619 12:13:37.199787 22323 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:37.199856 22323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:37.200007 22323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:37.201045 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "4a5f876efaeb4be6a6288aba655106f0"
format_stamp: "Formatted at 2026-06-19 12:13:37 on dist-test-slave-c910"
server_key: "8def09c175b35d2f1b0e832ec79cdc9e"
server_key_iv: "c66657d34610003a18de0e2efc14c692"
server_key_version: "encryptionkey@0"
I20260619 12:13:37.201402 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "4a5f876efaeb4be6a6288aba655106f0"
format_stamp: "Formatted at 2026-06-19 12:13:37 on dist-test-slave-c910"
server_key: "8def09c175b35d2f1b0e832ec79cdc9e"
server_key_iv: "c66657d34610003a18de0e2efc14c692"
server_key_version: "encryptionkey@0"
I20260619 12:13:37.202751 22323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:37.203500 22338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:37.203680 22323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:37.203779 22323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "4a5f876efaeb4be6a6288aba655106f0"
format_stamp: "Formatted at 2026-06-19 12:13:37 on dist-test-slave-c910"
server_key: "8def09c175b35d2f1b0e832ec79cdc9e"
server_key_iv: "c66657d34610003a18de0e2efc14c692"
server_key_version: "encryptionkey@0"
I20260619 12:13:37.203859 22323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:37.219111 22323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:37.221472 22323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:37.221639 22323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:37.225647 22323 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:36707
I20260619 12:13:37.225708 22390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:36707 every 8 connection(s)
I20260619 12:13:37.226091 22323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:37.226975 22391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:37.229252 22391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0: Bootstrap starting.
I20260619 12:13:37.229771 22391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:37.229997 22391 log.cc:824] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:37.230633 22391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0: No bootstrap required, opened a new log
I20260619 12:13:37.232246 22391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5f876efaeb4be6a6288aba655106f0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36707 } }
I20260619 12:13:37.232400 22391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:37.232430 22391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5f876efaeb4be6a6288aba655106f0, State: Initialized, Role: FOLLOWER
I20260619 12:13:37.232576 22391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5f876efaeb4be6a6288aba655106f0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36707 } }
I20260619 12:13:37.232656 22391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:37.232679 22391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:37.232717 22391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:37.233189 22391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5f876efaeb4be6a6288aba655106f0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36707 } }
I20260619 12:13:37.233269 22391 leader_election.cc:302] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a5f876efaeb4be6a6288aba655106f0; no voters:
I20260619 12:13:37.233397 22391 leader_election.cc:288] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:37.233462 22394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:37.233623 22394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [term 1 LEADER]: Becoming Leader. State: Replica: 4a5f876efaeb4be6a6288aba655106f0, State: Running, Role: LEADER
I20260619 12:13:37.233661 22391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:37.233711 22394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5f876efaeb4be6a6288aba655106f0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36707 } }
I20260619 12:13:37.234099 22396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a5f876efaeb4be6a6288aba655106f0. Latest consensus state: current_term: 1 leader_uuid: "4a5f876efaeb4be6a6288aba655106f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5f876efaeb4be6a6288aba655106f0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36707 } } }
I20260619 12:13:37.234239 22396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:37.234227 22395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a5f876efaeb4be6a6288aba655106f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5f876efaeb4be6a6288aba655106f0" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36707 } } }
I20260619 12:13:37.234335 22395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:37.234678 22404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:37.235292 22404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:37.236220 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22323
I20260619 12:13:37.236352 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:37.236567 804 external_mini_cluster.cc:1428] Setting key a7c523eb5f9977053124a904edb6f6b4
I20260619 12:13:37.236825 22404 catalog_manager.cc:1347] Generated new cluster ID: acaf78ece840464286769ee8ec35a87c
I20260619 12:13:37.236878 22404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:37.242889 22404 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:37.243474 22404 catalog_manager.cc:1504] Loading token signing keys...
Jun 19 12:13:37 dist-test-slave-c910 krb5kdc[22310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:37.250257 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:37.237189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53583 (local address 127.0.201.62:36707)
0619 12:13:37.239313 (+ 2124us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:37.239322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:37.239354 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:13:37.239464 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:37.239471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:37.239532 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:37.239662 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:37.240461 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:37.241912 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:37.243026 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:37.243687 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:37.243808 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:37.246632 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:37.246636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:37.246636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:37.246648 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:37.248566 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:37.249023 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:37.249026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:37.249026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:37.249124 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:37.249357 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:37.249358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:37.249358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:37.249619 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:37.249748 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:37.249926 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 12:13:37.250022 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2082,"thread_start_us":66,"threads_started":1}
I20260619 12:13:37.254458 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:37.258025 22404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a5f876efaeb4be6a6288aba655106f0: Generated new TSK 0
I20260619 12:13:37.258277 22404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:37.264006 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:37.256322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36375 (local address 127.0.201.62:36707)
0619 12:13:37.257276 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:37.257277 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:37.257286 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:37.257305 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:37.257306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:37.257317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:37.257376 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:37.257997 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:37.258974 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:37.259833 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:37.260066 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:37.260160 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:37.261405 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:37.261407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:37.261407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:37.261415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:37.262805 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:37.263175 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:37.263176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:37.263176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:37.263249 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:37.263482 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:37.263483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:37.263483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:37.263593 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:37.263715 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:37.263832 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:13:37.263895 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937}
I20260619 12:13:37.265854 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22323
W20260619 12:13:37.270414 22308 connection.cc:535] client connection to 127.0.201.62:36707 recv error: Network error: failed to read from TLS socket (remote: 127.0.201.62:36707): Connection reset by peer (error 104)
2026-06-19T12:13:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:37 dist-test-slave-c910 krb5kdc[22422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:37 dist-test-slave-c910 krb5kdc[22422](info): set up 2 sockets
Jun 19 12:13:37 dist-test-slave-c910 krb5kdc[22422](info): commencing operation
krb5kdc: starting...
W20260619 12:13:38.295557 804 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:38 dist-test-slave-c910 krb5kdc[22422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:38.414345 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36507
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:36507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:38.496440 22435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:38.496709 22435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:38.496749 22435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:38.497916 22435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:38.497989 22435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:38.498013 22435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:38.498035 22435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:38.498063 22435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:38.499569 22435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:36507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:38.499868 22435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:38.500108 22435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:38.504966 22440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:38.504969 22443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:38.505043 22441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:38.505395 22435 server_base.cc:1034] running on GCE node
I20260619 12:13:38.505654 22435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:38.505990 22435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:38.507136 22435 hybrid_clock.cc:648] HybridClock initialized: now 1781871218507120 us; error 34 us; skew 500 ppm
Jun 19 12:13:38 dist-test-slave-c910 krb5kdc[22422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:38.509938 22435 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:38.510457 22435 webserver.cc:460] Webserver started at http://127.0.201.62:36571/ using document root <none> and password file <none>
I20260619 12:13:38.510710 22435 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:38.510782 22435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:38.510931 22435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:38.512070 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "c4100bedf0f24a3886a1b11f3750e132"
format_stamp: "Formatted at 2026-06-19 12:13:38 on dist-test-slave-c910"
server_key: "94dd63d07969b7995954d0700e52c37f"
server_key_iv: "32d1ab72aec5eb9df1f50f04ce1f11ee"
server_key_version: "encryptionkey@0"
I20260619 12:13:38.512445 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "c4100bedf0f24a3886a1b11f3750e132"
format_stamp: "Formatted at 2026-06-19 12:13:38 on dist-test-slave-c910"
server_key: "94dd63d07969b7995954d0700e52c37f"
server_key_iv: "32d1ab72aec5eb9df1f50f04ce1f11ee"
server_key_version: "encryptionkey@0"
I20260619 12:13:38.513818 22435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:13:38.514611 22450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:38.514789 22435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:38.514897 22435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "c4100bedf0f24a3886a1b11f3750e132"
format_stamp: "Formatted at 2026-06-19 12:13:38 on dist-test-slave-c910"
server_key: "94dd63d07969b7995954d0700e52c37f"
server_key_iv: "32d1ab72aec5eb9df1f50f04ce1f11ee"
server_key_version: "encryptionkey@0"
I20260619 12:13:38.514976 22435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:38.528708 22435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:38.531450 22435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:38.531642 22435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:38.536410 22435 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:36507
I20260619 12:13:38.536443 22502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:36507 every 8 connection(s)
I20260619 12:13:38.536859 22435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:38.537798 22503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:38.539311 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22435
I20260619 12:13:38.539438 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:38.539654 804 external_mini_cluster.cc:1428] Setting key bef749fa53439db3737efa5a2478e955
I20260619 12:13:38.540536 22503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132: Bootstrap starting.
I20260619 12:13:38.541145 22503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:38.541518 22503 log.cc:824] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:38.542212 22503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132: No bootstrap required, opened a new log
I20260619 12:13:38.544626 22503 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4100bedf0f24a3886a1b11f3750e132" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36507 } }
I20260619 12:13:38.544827 22503 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:38.544870 22503 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4100bedf0f24a3886a1b11f3750e132, State: Initialized, Role: FOLLOWER
I20260619 12:13:38.545063 22503 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4100bedf0f24a3886a1b11f3750e132" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36507 } }
I20260619 12:13:38.545137 22503 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:38.545159 22503 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:38.545228 22503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:38.545763 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:38.540266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44129 (local address 127.0.201.62:36507)
0619 12:13:38.540461 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:38.540470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:38.540499 (+ 29us) server_negotiation.cc:407] Connection header received
0619 12:13:38.541003 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:38.541010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:38.541063 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:38.541200 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:38.541491 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:38.542523 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:38.543676 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:38.544086 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:38.544176 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:38.544982 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:38.544987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:38.544987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:38.545232 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:38.545274 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:38.545290 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:38.545379 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:38.545531 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 12:13:38.545564 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260619 12:13:38.545934 22503 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4100bedf0f24a3886a1b11f3750e132" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36507 } }
I20260619 12:13:38.546075 22503 leader_election.cc:302] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4100bedf0f24a3886a1b11f3750e132; no voters:
I20260619 12:13:38.546305 22503 leader_election.cc:288] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:38.546330 22508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:38.546567 22508 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [term 1 LEADER]: Becoming Leader. State: Replica: c4100bedf0f24a3886a1b11f3750e132, State: Running, Role: LEADER
I20260619 12:13:38.546640 22503 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:38.546720 22508 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4100bedf0f24a3886a1b11f3750e132" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36507 } }
I20260619 12:13:38.546765 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:38.547156 22508 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4100bedf0f24a3886a1b11f3750e132. Latest consensus state: current_term: 1 leader_uuid: "c4100bedf0f24a3886a1b11f3750e132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4100bedf0f24a3886a1b11f3750e132" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36507 } } }
I20260619 12:13:38.547169 22509 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4100bedf0f24a3886a1b11f3750e132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4100bedf0f24a3886a1b11f3750e132" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 36507 } } }
I20260619 12:13:38.547266 22508 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:38.547308 22509 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:38.548058 22517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:38.548595 22517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:38.550272 22517 catalog_manager.cc:1347] Generated new cluster ID: 88c9000cdd3e46f49784856a64e0b2ef
I20260619 12:13:38.550328 22517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:38.552448 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:38.548506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53931 (local address 127.0.201.62:36507)
0619 12:13:38.548540 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:38.548542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:38.548733 (+ 191us) server_negotiation.cc:407] Connection header received
0619 12:13:38.548810 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:38.548810 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:38.548821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:38.548881 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:38.549057 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:38.549815 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:38.550864 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:38.551160 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:38.551248 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:38.552134 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:38.552135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:38.552136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:38.552143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:38.552158 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:38.552162 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:38.552195 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:38.552355 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 12:13:38.552368 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 12:13:38.561759 22517 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:38.562296 22517 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:38.573707 22517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4100bedf0f24a3886a1b11f3750e132: Generated new TSK 0
I20260619 12:13:38.573988 22517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:38.581619 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22435
W20260619 12:13:38.587960 22420 connection.cc:535] client connection to 127.0.201.62:36507 recv error: Network error: failed to read from TLS socket (remote: 127.0.201.62:36507): Connection reset by peer (error 104)
2026-06-19T12:13:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:38 dist-test-slave-c910 krb5kdc[22534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:38 dist-test-slave-c910 krb5kdc[22534](info): set up 2 sockets
Jun 19 12:13:38 dist-test-slave-c910 krb5kdc[22534](info): commencing operation
krb5kdc: starting...
W20260619 12:13:39.607379 804 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:39 dist-test-slave-c910 krb5kdc[22534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:39.733690 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:43817
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:43817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:39.816156 22547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:39.816373 22547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:39.816397 22547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:39.817574 22547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:39.817623 22547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:39.817638 22547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:39.817651 22547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:39.817708 22547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:39.819300 22547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:43817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:43817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:39.819571 22547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:39.819823 22547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:39.824829 22547 server_base.cc:1034] running on GCE node
W20260619 12:13:39.824818 22553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:39.824818 22555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:39.824909 22552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:39.825309 22547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:39.825713 22547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:39.826917 22547 hybrid_clock.cc:648] HybridClock initialized: now 1781871219826895 us; error 38 us; skew 500 ppm
Jun 19 12:13:39 dist-test-slave-c910 krb5kdc[22534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871219, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:39.829559 22547 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:39.830056 22547 webserver.cc:460] Webserver started at http://127.0.201.62:37177/ using document root <none> and password file <none>
I20260619 12:13:39.830286 22547 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:39.830354 22547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:39.830513 22547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:39.831673 22547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "0bbf564a48be4d80a008e3f94a404394"
format_stamp: "Formatted at 2026-06-19 12:13:39 on dist-test-slave-c910"
server_key: "b7fab059b810f6f8cb9f8179a0866cf6"
server_key_iv: "03a18828635b2f1b1eb6248a7ecb7064"
server_key_version: "encryptionkey@0"
I20260619 12:13:39.832041 22547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "0bbf564a48be4d80a008e3f94a404394"
format_stamp: "Formatted at 2026-06-19 12:13:39 on dist-test-slave-c910"
server_key: "b7fab059b810f6f8cb9f8179a0866cf6"
server_key_iv: "03a18828635b2f1b1eb6248a7ecb7064"
server_key_version: "encryptionkey@0"
I20260619 12:13:39.833446 22547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 12:13:39.834317 22562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:39.834599 22547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:39.834746 22547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "0bbf564a48be4d80a008e3f94a404394"
format_stamp: "Formatted at 2026-06-19 12:13:39 on dist-test-slave-c910"
server_key: "b7fab059b810f6f8cb9f8179a0866cf6"
server_key_iv: "03a18828635b2f1b1eb6248a7ecb7064"
server_key_version: "encryptionkey@0"
I20260619 12:13:39.834823 22547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:39.838197 22547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:39.840422 22547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:39.840583 22547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:39.844415 22547 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:43817
I20260619 12:13:39.844475 22614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:43817 every 8 connection(s)
I20260619 12:13:39.844831 22547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:39.845856 22615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:39.848187 22615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394: Bootstrap starting.
I20260619 12:13:39.848706 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22547
I20260619 12:13:39.848832 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:39.848881 22615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:39.849082 804 external_mini_cluster.cc:1428] Setting key 9dd09a73923adcd2e1b5ab538aac46dc
I20260619 12:13:39.849224 22615 log.cc:824] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:39.850095 22615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394: No bootstrap required, opened a new log
I20260619 12:13:39.851783 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:39.849638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54301 (local address 127.0.201.62:43817)
0619 12:13:39.849805 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:39.849815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:39.849847 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:13:39.850432 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:39.850441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:39.850478 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:39.850634 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:39.851335 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:39.851340 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:39.851340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:39.851345 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:39.851390 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:39.851405 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:39.851496 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:39.851528 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 12:13:39.851571 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260619 12:13:39.852900 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:39.852809 22615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbf564a48be4d80a008e3f94a404394" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43817 } }
I20260619 12:13:39.852972 22615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:39.853036 22615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbf564a48be4d80a008e3f94a404394, State: Initialized, Role: FOLLOWER
I20260619 12:13:39.853161 22615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbf564a48be4d80a008e3f94a404394" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43817 } }
I20260619 12:13:39.853255 22615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:39.853297 22615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:39.853348 22615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:39.853938 22615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbf564a48be4d80a008e3f94a404394" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43817 } }
I20260619 12:13:39.854061 22615 leader_election.cc:302] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bbf564a48be4d80a008e3f94a404394; no voters:
I20260619 12:13:39.854290 22615 leader_election.cc:288] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:39.854339 22624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:39.854517 22624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [term 1 LEADER]: Becoming Leader. State: Replica: 0bbf564a48be4d80a008e3f94a404394, State: Running, Role: LEADER
I20260619 12:13:39.854616 22615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:39.854672 22624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbf564a48be4d80a008e3f94a404394" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43817 } }
I20260619 12:13:39.855094 22627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bbf564a48be4d80a008e3f94a404394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbf564a48be4d80a008e3f94a404394" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43817 } } }
I20260619 12:13:39.855270 22627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:39.855249 22626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bbf564a48be4d80a008e3f94a404394. Latest consensus state: current_term: 1 leader_uuid: "0bbf564a48be4d80a008e3f94a404394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbf564a48be4d80a008e3f94a404394" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 43817 } } }
I20260619 12:13:39.855376 22626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:39.855762 22634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:39.856027 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:39.854401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58209 (local address 127.0.201.62:43817)
0619 12:13:39.854448 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:39.854451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:39.854588 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:13:39.854893 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:39.854894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:39.854899 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:39.854928 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:39.855665 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:39.855666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:39.855667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:39.855672 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:39.855687 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:39.855692 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:39.855744 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:39.855950 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 12:13:39.855963 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:13:39.856632 22634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:39.858198 22634 catalog_manager.cc:1347] Generated new cluster ID: 863633fe23f6436c99ca83dca616ea04
I20260619 12:13:39.858260 22634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:39.877609 22634 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:39.877862 22634 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:39.886749 22634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bbf564a48be4d80a008e3f94a404394: Generated new TSK 0
I20260619 12:13:39.886968 22634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:39.896518 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22547
2026-06-19T12:13:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:39 dist-test-slave-c910 krb5kdc[22646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:39 dist-test-slave-c910 krb5kdc[22646](info): set up 2 sockets
Jun 19 12:13:39 dist-test-slave-c910 krb5kdc[22646](info): commencing operation
krb5kdc: starting...
W20260619 12:13:40.939388 804 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:41 dist-test-slave-c910 krb5kdc[22646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:41.058388 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45625
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:45625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:41.139421 22659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:41.139643 22659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:41.139668 22659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:41.140887 22659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:41.140935 22659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:41.140956 22659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:41.140970 22659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:41.141012 22659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:41.142518 22659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:45625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:45625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:41.142773 22659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:41.143021 22659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:41.147959 22667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:41.147933 22665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:41.148005 22659 server_base.cc:1034] running on GCE node
W20260619 12:13:41.147933 22664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:41.148424 22659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:41.148736 22659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:41.149909 22659 hybrid_clock.cc:648] HybridClock initialized: now 1781871221149889 us; error 42 us; skew 500 ppm
Jun 19 12:13:41 dist-test-slave-c910 krb5kdc[22646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871221, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:41.152531 22659 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:41.153053 22659 webserver.cc:460] Webserver started at http://127.0.201.62:40739/ using document root <none> and password file <none>
I20260619 12:13:41.153317 22659 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:41.153366 22659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:41.153535 22659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:41.154619 22659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "462c7792ab2947e09955a01607bb5753"
format_stamp: "Formatted at 2026-06-19 12:13:41 on dist-test-slave-c910"
server_key: "58408f1549f1deb7b8fef77dec08e4af"
server_key_iv: "35be80fc96035627d597548922a0bf98"
server_key_version: "encryptionkey@0"
I20260619 12:13:41.154978 22659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "462c7792ab2947e09955a01607bb5753"
format_stamp: "Formatted at 2026-06-19 12:13:41 on dist-test-slave-c910"
server_key: "58408f1549f1deb7b8fef77dec08e4af"
server_key_iv: "35be80fc96035627d597548922a0bf98"
server_key_version: "encryptionkey@0"
I20260619 12:13:41.156380 22659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:13:41.157112 22674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:41.157287 22659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:41.157368 22659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "462c7792ab2947e09955a01607bb5753"
format_stamp: "Formatted at 2026-06-19 12:13:41 on dist-test-slave-c910"
server_key: "58408f1549f1deb7b8fef77dec08e4af"
server_key_iv: "35be80fc96035627d597548922a0bf98"
server_key_version: "encryptionkey@0"
I20260619 12:13:41.157456 22659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:41.173105 22659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:41.175312 22659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:41.175464 22659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:41.179514 22659 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:45625
I20260619 12:13:41.179569 22726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:45625 every 8 connection(s)
I20260619 12:13:41.179919 22659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:41.180820 22727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:41.183025 22727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753: Bootstrap starting.
I20260619 12:13:41.183390 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22659
I20260619 12:13:41.183485 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:41.183681 804 external_mini_cluster.cc:1428] Setting key 726aa53f63dbf49d92d4dd57c622ce85
I20260619 12:13:41.183737 22727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:41.184065 22727 log.cc:824] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:41.184904 22727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753: No bootstrap required, opened a new log
I20260619 12:13:41.187278 22727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462c7792ab2947e09955a01607bb5753" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45625 } }
I20260619 12:13:41.187484 22727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:41.187523 22727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462c7792ab2947e09955a01607bb5753, State: Initialized, Role: FOLLOWER
I20260619 12:13:41.187654 22727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462c7792ab2947e09955a01607bb5753" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45625 } }
I20260619 12:13:41.187731 22727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:41.187759 22727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:41.187795 22727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:41.188452 22727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462c7792ab2947e09955a01607bb5753" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45625 } }
I20260619 12:13:41.188568 22727 leader_election.cc:302] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 462c7792ab2947e09955a01607bb5753; no voters:
I20260619 12:13:41.188758 22727 leader_election.cc:288] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:41.188786 22732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:41.189013 22727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:41.189265 22732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [term 1 LEADER]: Becoming Leader. State: Replica: 462c7792ab2947e09955a01607bb5753, State: Running, Role: LEADER
I20260619 12:13:41.189384 22732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462c7792ab2947e09955a01607bb5753" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45625 } }
I20260619 12:13:41.189743 22732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 462c7792ab2947e09955a01607bb5753. Latest consensus state: current_term: 1 leader_uuid: "462c7792ab2947e09955a01607bb5753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462c7792ab2947e09955a01607bb5753" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45625 } } }
I20260619 12:13:41.189838 22732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:41.189752 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "462c7792ab2947e09955a01607bb5753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462c7792ab2947e09955a01607bb5753" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 45625 } } }
I20260619 12:13:41.189925 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:41.190604 22741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 12:13:41 dist-test-slave-c910 krb5kdc[22646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:41.191172 22741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:41.193014 22741 catalog_manager.cc:1347] Generated new cluster ID: 5c22482bacf6426e961194b862647653
I20260619 12:13:41.193069 22741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:41.194991 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:41.184253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35427 (local address 127.0.201.62:45625)
0619 12:13:41.184456 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:41.184466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:41.184501 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:13:41.185045 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:41.185061 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:41.185112 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:41.185231 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:41.186074 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:41.186970 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:41.188072 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:41.188481 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:41.188611 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:41.191566 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:41.191570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:41.191570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:41.191584 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:41.193464 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:41.193838 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:41.193841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:41.193842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:41.193931 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:41.194139 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:41.194140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:41.194140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:41.194382 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:41.194560 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:41.194637 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 12:13:41.194792 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260619 12:13:41.195943 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:41.204314 22741 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:41.205020 22741 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:41.205906 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:41.199582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33971 (local address 127.0.201.62:45625)
0619 12:13:41.199619 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:41.199622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:41.199832 (+ 210us) server_negotiation.cc:407] Connection header received
0619 12:13:41.199921 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:41.199922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:41.199932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:41.199988 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:41.200518 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:41.201010 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:41.201726 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:41.201938 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:41.202013 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:41.203270 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:41.203271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:41.203271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:41.203278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:41.204746 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:41.205148 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:41.205149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:41.205150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:41.205210 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:41.205385 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:41.205386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:41.205386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:41.205489 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:41.205587 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:41.205704 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:13:41.205805 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 12:13:41.214591 22741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 462c7792ab2947e09955a01607bb5753: Generated new TSK 0
I20260619 12:13:41.214820 22741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:41.217384 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22659
2026-06-19T12:13:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:41 dist-test-slave-c910 krb5kdc[22758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:41 dist-test-slave-c910 krb5kdc[22758](info): set up 2 sockets
Jun 19 12:13:41 dist-test-slave-c910 krb5kdc[22758](info): commencing operation
krb5kdc: starting...
W20260619 12:13:42.247453 804 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:42 dist-test-slave-c910 krb5kdc[22758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:42.371168 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:35747
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:42.454867 22771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:42.455106 22771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:42.455132 22771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:42.456341 22771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:42.456395 22771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:42.456413 22771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:42.456427 22771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:42.456441 22771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:42.457892 22771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:42.458117 22771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:42.458308 22771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:42.463717 22771 server_base.cc:1034] running on GCE node
W20260619 12:13:42.463701 22777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:42.463701 22776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:42.463850 22779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:42.464349 22771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:42.464684 22771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:42.465849 22771 hybrid_clock.cc:648] HybridClock initialized: now 1781871222465848 us; error 44 us; skew 500 ppm
Jun 19 12:13:42 dist-test-slave-c910 krb5kdc[22758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871222, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:42.474406 22771 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:42.474916 22771 webserver.cc:460] Webserver started at http://127.0.201.62:32877/ using document root <none> and password file <none>
I20260619 12:13:42.475173 22771 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:42.475312 22771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:42.475476 22771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:42.476922 22771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "75cf85aa629845d693c4625e96bfaea4"
format_stamp: "Formatted at 2026-06-19 12:13:42 on dist-test-slave-c910"
server_key: "7092d2e3057bfaa4e9990c4d53abcba8"
server_key_iv: "f9e1315bb974c1faf93686df61df5e09"
server_key_version: "encryptionkey@0"
I20260619 12:13:42.477319 22771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "75cf85aa629845d693c4625e96bfaea4"
format_stamp: "Formatted at 2026-06-19 12:13:42 on dist-test-slave-c910"
server_key: "7092d2e3057bfaa4e9990c4d53abcba8"
server_key_iv: "f9e1315bb974c1faf93686df61df5e09"
server_key_version: "encryptionkey@0"
I20260619 12:13:42.478835 22771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:42.479758 22786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:42.479970 22771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 12:13:42.480084 22771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "75cf85aa629845d693c4625e96bfaea4"
format_stamp: "Formatted at 2026-06-19 12:13:42 on dist-test-slave-c910"
server_key: "7092d2e3057bfaa4e9990c4d53abcba8"
server_key_iv: "f9e1315bb974c1faf93686df61df5e09"
server_key_version: "encryptionkey@0"
I20260619 12:13:42.480186 22771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:42.501578 22771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:42.503746 22771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:42.503866 22771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:42.507736 22771 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:35747
I20260619 12:13:42.507783 22838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:35747 every 8 connection(s)
I20260619 12:13:42.508167 22771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:42.509105 22839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:42.511446 22839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4: Bootstrap starting.
I20260619 12:13:42.512007 22839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:42.512288 22839 log.cc:824] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:42.512989 22839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4: No bootstrap required, opened a new log
I20260619 12:13:42.514503 22839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75cf85aa629845d693c4625e96bfaea4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35747 } }
I20260619 12:13:42.514643 22839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:42.514667 22839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75cf85aa629845d693c4625e96bfaea4, State: Initialized, Role: FOLLOWER
I20260619 12:13:42.514854 22839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75cf85aa629845d693c4625e96bfaea4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35747 } }
I20260619 12:13:42.514936 22839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:42.514978 22839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:42.515027 22839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:42.515615 22839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75cf85aa629845d693c4625e96bfaea4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35747 } }
I20260619 12:13:42.515699 22839 leader_election.cc:302] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75cf85aa629845d693c4625e96bfaea4; no voters:
I20260619 12:13:42.515877 22839 leader_election.cc:288] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:42.515905 22842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:42.516091 22842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [term 1 LEADER]: Becoming Leader. State: Replica: 75cf85aa629845d693c4625e96bfaea4, State: Running, Role: LEADER
I20260619 12:13:42.516145 22839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:42.516180 22842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75cf85aa629845d693c4625e96bfaea4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35747 } }
I20260619 12:13:42.516454 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22771
I20260619 12:13:42.516578 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:42.516817 804 external_mini_cluster.cc:1428] Setting key 5ab8f8c92f51d08ec3b326677981e182
I20260619 12:13:42.517407 22843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75cf85aa629845d693c4625e96bfaea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75cf85aa629845d693c4625e96bfaea4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35747 } } }
I20260619 12:13:42.517511 22843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:42.517719 22844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75cf85aa629845d693c4625e96bfaea4. Latest consensus state: current_term: 1 leader_uuid: "75cf85aa629845d693c4625e96bfaea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75cf85aa629845d693c4625e96bfaea4" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 35747 } } }
I20260619 12:13:42.517850 22844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4 [sys.catalog]: This master's current role is: LEADER
W20260619 12:13:42.518129 22859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:42.518183 22859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:42.518246 22860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:42.518412 22860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:42.519815 22860 catalog_manager.cc:1347] Generated new cluster ID: 72d31d700b9b41f4b505987700a77d8d
I20260619 12:13:42.519872 22860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:42.523061 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:42.517377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48073 (local address 127.0.201.62:35747)
0619 12:13:42.517629 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:42.517640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:42.517686 (+ 46us) server_negotiation.cc:407] Connection header received
0619 12:13:42.517910 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:42.517919 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:42.517980 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:42.518126 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:42.518396 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:42.519803 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:42.520899 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:42.521542 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:42.521666 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:42.522374 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:42.522377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:42.522378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:42.522589 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:42.522628 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:42.522642 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:42.522713 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:42.522782 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:13:42.522846 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260619 12:13:42.524036 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:42.528880 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:42.525446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52799 (local address 127.0.201.62:35747)
0619 12:13:42.525481 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:42.525483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:42.525540 (+ 57us) server_negotiation.cc:407] Connection header received
0619 12:13:42.525594 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:42.525595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:42.525605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:42.525674 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:42.525793 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:42.526526 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:42.527549 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:42.527875 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:42.527948 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:42.528626 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:42.528627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:42.528628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:42.528632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:42.528648 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:42.528652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:42.528712 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:42.528779 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 12:13:42.528803 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:42.532814 22860 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:42.533530 22860 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:42.540230 22860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75cf85aa629845d693c4625e96bfaea4: Generated new TSK 0
I20260619 12:13:42.540426 22860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:42.543622 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22771
2026-06-19T12:13:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:42 dist-test-slave-c910 krb5kdc[22870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:42 dist-test-slave-c910 krb5kdc[22870](info): set up 2 sockets
Jun 19 12:13:42 dist-test-slave-c910 krb5kdc[22870](info): commencing operation
krb5kdc: starting...
W20260619 12:13:43.571571 804 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:43 dist-test-slave-c910 krb5kdc[22870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:43.690317 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42617
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:41751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:43.776218 22883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:43.776448 22883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:43.776471 22883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:43.777567 22883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:43.777606 22883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:43.777619 22883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:43.777632 22883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:43.777644 22883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:43.779069 22883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:41751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:43.779354 22883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:43.779543 22883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:43.784466 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:43.784492 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:43.784744 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:43.784745 22883 server_base.cc:1034] running on GCE node
I20260619 12:13:43.785037 22883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:43.785310 22883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:43.786450 22883 hybrid_clock.cc:648] HybridClock initialized: now 1781871223786416 us; error 49 us; skew 500 ppm
Jun 19 12:13:43 dist-test-slave-c910 krb5kdc[22870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871223, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:43.789014 22883 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:43.789410 22883 webserver.cc:460] Webserver started at http://127.0.201.62:46801/ using document root <none> and password file <none>
I20260619 12:13:43.789605 22883 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:43.789645 22883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:43.789736 22883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:43.790759 22883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "0869b42c54d24fb19e5059e97a89a873"
format_stamp: "Formatted at 2026-06-19 12:13:43 on dist-test-slave-c910"
server_key: "6902aed0e0cd1b54f80b4d0a7e2620c6"
server_key_iv: "7803c48434acfe9c3e9bce9e28e9a430"
server_key_version: "encryptionkey@0"
I20260619 12:13:43.791052 22883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "0869b42c54d24fb19e5059e97a89a873"
format_stamp: "Formatted at 2026-06-19 12:13:43 on dist-test-slave-c910"
server_key: "6902aed0e0cd1b54f80b4d0a7e2620c6"
server_key_iv: "7803c48434acfe9c3e9bce9e28e9a430"
server_key_version: "encryptionkey@0"
I20260619 12:13:43.792392 22883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:13:43.793093 22898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:43.793267 22883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:43.793373 22883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "0869b42c54d24fb19e5059e97a89a873"
format_stamp: "Formatted at 2026-06-19 12:13:43 on dist-test-slave-c910"
server_key: "6902aed0e0cd1b54f80b4d0a7e2620c6"
server_key_iv: "7803c48434acfe9c3e9bce9e28e9a430"
server_key_version: "encryptionkey@0"
I20260619 12:13:43.793449 22883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:43.796871 22883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:43.798707 22883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:43.798866 22883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:43.802740 22883 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42617
I20260619 12:13:43.802772 22950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42617 every 8 connection(s)
I20260619 12:13:43.803107 22883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:43.804139 22951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:43.806463 22951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873: Bootstrap starting.
I20260619 12:13:43.807050 22951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:43.807404 22951 log.cc:824] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:43.808090 22951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873: No bootstrap required, opened a new log
I20260619 12:13:43.809266 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22883
I20260619 12:13:43.809387 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:43.809572 804 external_mini_cluster.cc:1428] Setting key 432884facae7317ed2216720540c0aec
I20260619 12:13:43.809801 22951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869b42c54d24fb19e5059e97a89a873" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42617 } }
I20260619 12:13:43.809976 22951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:13:43.810019 22951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0869b42c54d24fb19e5059e97a89a873, State: Initialized, Role: FOLLOWER
I20260619 12:13:43.810190 22951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869b42c54d24fb19e5059e97a89a873" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42617 } }
I20260619 12:13:43.810295 22951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:13:43.810331 22951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:13:43.810382 22951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:43.811151 22951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869b42c54d24fb19e5059e97a89a873" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42617 } }
I20260619 12:13:43.811302 22951 leader_election.cc:302] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0869b42c54d24fb19e5059e97a89a873; no voters:
I20260619 12:13:43.811473 22951 leader_election.cc:288] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:13:43.811506 22956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:43.811704 22956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [term 1 LEADER]: Becoming Leader. State: Replica: 0869b42c54d24fb19e5059e97a89a873, State: Running, Role: LEADER
I20260619 12:13:43.811750 22951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:43.811863 22956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869b42c54d24fb19e5059e97a89a873" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42617 } }
I20260619 12:13:43.812029 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:43.810093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54605 (local address 127.0.201.62:42617)
0619 12:13:43.810301 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:43.810310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:43.810346 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:13:43.810474 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:43.810481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:43.810513 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:43.810663 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:43.811409 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:43.811412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:43.811413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:43.811418 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:43.811464 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:43.811480 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:43.811559 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:43.811787 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 12:13:43.811836 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260619 12:13:43.812220 22956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0869b42c54d24fb19e5059e97a89a873. Latest consensus state: current_term: 1 leader_uuid: "0869b42c54d24fb19e5059e97a89a873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869b42c54d24fb19e5059e97a89a873" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42617 } } }
I20260619 12:13:43.812280 22956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:43.812994 22962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:43.813086 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:43.813535 22962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:43.814494 22957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0869b42c54d24fb19e5059e97a89a873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0869b42c54d24fb19e5059e97a89a873" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42617 } } }
I20260619 12:13:43.814649 22957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:43.815026 22962 catalog_manager.cc:1347] Generated new cluster ID: af5a394822ba4ee59e239ba585a948de
I20260619 12:13:43.815078 22962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:43.816345 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:43.814565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35705 (local address 127.0.201.62:42617)
0619 12:13:43.814598 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:43.814600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:43.815349 (+ 749us) server_negotiation.cc:407] Connection header received
0619 12:13:43.815423 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:43.815424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:43.815431 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:43.815486 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:43.816110 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:43.816112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:43.816112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:13:43.816116 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:43.816130 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:13:43.816134 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:43.816191 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:43.816245 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:13:43.816260 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 12:13:43.826890 22962 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:43.827086 22962 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:43.832685 22962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0869b42c54d24fb19e5059e97a89a873: Generated new TSK 0
I20260619 12:13:43.832894 22962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:13:43.837924 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22883
2026-06-19T12:13:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1295 ms)
[----------] 18 tests from AuthTokenIssuingTest (23620 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:43 dist-test-slave-c910 krb5kdc[22982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:43 dist-test-slave-c910 krb5kdc[22982](info): set up 2 sockets
Jun 19 12:13:43 dist-test-slave-c910 krb5kdc[22982](info): commencing operation
krb5kdc: starting...
W20260619 12:13:44.871421 804 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:44 dist-test-slave-c910 krb5kdc[22982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:44.990347 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44131
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:44131,127.0.201.61:44453,127.0.201.60:41597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:45.072367 22995 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:13:45.072630 22995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:45.072667 22995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:45.072710 22995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:45.073887 22995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:45.073966 22995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:45.073990 22995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:45.074019 22995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:45.074046 22995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:45.075588 22995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:44131,127.0.201.61:44453,127.0.201.60:41597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:45.075870 22995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:45.076105 22995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:45.081169 22995 server_base.cc:1034] running on GCE node
W20260619 12:13:45.081146 23001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:45.081146 23003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:45.081130 23000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:45.081735 22995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:45.082077 22995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:45.083285 22995 hybrid_clock.cc:648] HybridClock initialized: now 1781871225083331 us; error 92 us; skew 500 ppm
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:45.085848 22995 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:45.086344 22995 webserver.cc:460] Webserver started at http://127.0.201.62:45693/ using document root <none> and password file <none>
I20260619 12:13:45.086678 22995 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:45.086750 22995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:45.086884 22995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:45.088024 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "ff7c246155344cb1833349b33eefcc10"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "d586e06d8683cb80580f9a3af363c4c5"
server_key_iv: "c342e94e3dcfe5209a47a6090cda251f"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.088411 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "ff7c246155344cb1833349b33eefcc10"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "d586e06d8683cb80580f9a3af363c4c5"
server_key_iv: "c342e94e3dcfe5209a47a6090cda251f"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.089782 22995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 12:13:45.090526 23010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:45.090711 22995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:45.090818 22995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "ff7c246155344cb1833349b33eefcc10"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "d586e06d8683cb80580f9a3af363c4c5"
server_key_iv: "c342e94e3dcfe5209a47a6090cda251f"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.090909 22995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:45.103340 22995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:45.105844 22995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:45.106024 22995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:45.110472 22995 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:44131
I20260619 12:13:45.110505 23062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:44131 every 8 connection(s)
I20260619 12:13:45.110889 22995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:45.111927 23063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:45.113500 23063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:45.115423 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 22995
I20260619 12:13:45.115599 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:45.115798 804 external_mini_cluster.cc:1428] Setting key ffacca47aca9e1aa7225b010d949eeef
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.61@KRBTEST.COM; defaulting to no policy
I20260619 12:13:45.124167 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.114191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44131 (local address 127.0.0.1:36154)
0619 12:13:45.114427 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.114470 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.114966 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.115299 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.115301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.115618 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.115819 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.115823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.116849 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.116850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.117512 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.117514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.118049 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.118051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.118114 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.118717 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.118727 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.120414 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.122520 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.122527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.122529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.122754 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.123025 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.123025 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.123026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.123109 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.123615 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.123620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.123691 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.123775 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.123910 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
I20260619 12:13:45.124172 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.114261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.0.201.62:44131)
0619 12:13:45.114436 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.114445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.114570 (+ 125us) server_negotiation.cc:407] Connection header received
0619 12:13:45.115123 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.115125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.115144 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.115289 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.115931 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.116740 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.117613 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.117929 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.118068 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.120505 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.120513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.120514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.120522 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.122401 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.122843 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.122846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.122847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.122947 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.123223 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.123224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.123224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.123529 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.123633 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.123841 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.123987 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260619 12:13:45.125147 23063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:45.125504 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.125381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34868)
0619 12:13:45.125436 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.125483 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:13:45.125656 23011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
I20260619 12:13:45.126175 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.126029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34876)
0619 12:13:45.126112 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.126146 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 12:13:45.126288 23063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:44453: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab.
Entry for principal kudu/127.0.201.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab.
I20260619 12:13:45.140185 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:44453
--webserver_interface=127.0.201.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab
--principal=kudu/127.0.201.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:44131,127.0.201.61:44453,127.0.201.60:41597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:13:45.175509 23063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } attempt: 1
I20260619 12:13:45.175993 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.175861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34884)
0619 12:13:45.175924 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.175965 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 12:13:45.176353 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.176261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34890)
0619 12:13:45.176305 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.176329 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 12:13:45.176438 23063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:44453: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
W20260619 12:13:45.222883 23070 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:13:45.223151 23070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:45.223183 23070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:45.223238 23070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:45.224407 23070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:45.224474 23070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:45.224498 23070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:45.224527 23070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:45.224560 23070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:45.226043 23070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:44131,127.0.201.61:44453,127.0.201.60:41597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab
--principal=kudu/127.0.201.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:44453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.201.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:45.226322 23070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:45.226548 23070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:45.231431 23075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:45.231458 23070 server_base.cc:1034] running on GCE node
W20260619 12:13:45.231431 23076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:45.231463 23078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:45.231884 23070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:45.232188 23070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:45.233371 23070 hybrid_clock.cc:648] HybridClock initialized: now 1781871225233362 us; error 45 us; skew 500 ppm
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:45.235916 23070 init.cc:377] Logged in from keytab as kudu/127.0.201.61@KRBTEST.COM (short username kudu)
I20260619 12:13:45.236339 23070 webserver.cc:460] Webserver started at http://127.0.201.61:33293/ using document root <none> and password file <none>
I20260619 12:13:45.236580 23070 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:45.236649 23070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:45.236792 23070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:45.237848 23070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data/instance:
uuid: "0f36d16f851846f48450e7ba49e59c59"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "65eb4ad129b3e518b73652a37a65da2b"
server_key_iv: "9f93d844fe351a53f44c22d2e5353199"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.238193 23070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal/instance:
uuid: "0f36d16f851846f48450e7ba49e59c59"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "65eb4ad129b3e518b73652a37a65da2b"
server_key_iv: "9f93d844fe351a53f44c22d2e5353199"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.239542 23070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:45.240273 23085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:45.240442 23070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:45.240545 23070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal
uuid: "0f36d16f851846f48450e7ba49e59c59"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "65eb4ad129b3e518b73652a37a65da2b"
server_key_iv: "9f93d844fe351a53f44c22d2e5353199"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.240653 23070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:45.244565 23063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } attempt: 2
I20260619 12:13:45.244994 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.244857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34894)
0619 12:13:45.244917 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.244961 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:13:45.245394 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.245254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34902)
0619 12:13:45.245331 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.245368 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 12:13:45.245537 23063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:44453: Network error: Client connection negotiation failed: client connection to 127.0.201.61:44453: connect: Connection refused (error 111)
I20260619 12:13:45.268632 23070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:45.271041 23070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:45.271205 23070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:45.275326 23070 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.61:44453
I20260619 12:13:45.275379 23137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.61:44453 every 8 connection(s)
I20260619 12:13:45.275724 23070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
I20260619 12:13:45.276726 23138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:45.278331 23138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:45.285136 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 23070
I20260619 12:13:45.285255 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-1/wal/instance
I20260619 12:13:45.285446 804 external_mini_cluster.cc:1428] Setting key 4fc160fb0399cf329d1c7889504ff001
I20260619 12:13:45.289211 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.279113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.0.201.62:44131)
0619 12:13:45.279174 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.279176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.279426 (+ 250us) server_negotiation.cc:407] Connection header received
0619 12:13:45.279934 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.279936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.279947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.280012 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.280633 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.281133 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.282139 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.282359 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.282446 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.284599 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.284601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.284601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.284609 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.287396 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.287983 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.287984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.287985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.288061 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.288382 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.288383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.288383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.288543 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.288611 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.288962 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.289100 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:13:45.289383 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.279081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44131 (local address 127.0.0.1:36164)
0619 12:13:45.279296 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.279332 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.279825 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.280086 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.280090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.280344 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.280535 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.280539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.281250 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.281251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.282046 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.282047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.282464 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.282466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.282590 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.283070 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.283078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.284443 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.287494 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.287501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.287503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.287888 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.288139 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.288139 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.288140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.288323 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.288618 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.288625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.288798 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.288913 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.289099 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":136,"threads_started":1}
I20260619 12:13:45.289958 23138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.201.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.60@KRBTEST.COM" created.
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:45.303936 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.295449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34904)
0619 12:13:45.295528 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.295541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.295603 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.296379 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.296380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.296524 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.296580 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.296582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.297786 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.297788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.298648 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.298650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.299124 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.299124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.299189 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.299659 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.299664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.301069 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.302858 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.302859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.302860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.303023 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.303258 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.303259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.303259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.303314 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.303602 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.303603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.303640 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.303713 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.303817 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 12:13:45.304008 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.295874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34904 (local address 127.0.201.61:44453)
0619 12:13:45.296184 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.296189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.296216 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:13:45.296247 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.296249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.296264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.296342 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.296689 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.297693 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.298732 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.299041 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.299128 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.301166 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.301174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.301174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.301181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.302732 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.303113 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.303117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.303117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.303169 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.303393 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.303394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.303394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.303537 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.303623 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.303750 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.303887 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":259,"thread_start_us":103,"threads_started":1}
I20260619 12:13:45.305071 23138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:45.305408 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.305323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37374)
0619 12:13:45.305350 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.305389 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 12:13:45.305539 23087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
I20260619 12:13:45.305956 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.305873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37386)
0619 12:13:45.305917 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.305940 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 12:13:45.306074 23138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41597: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab.
Entry for principal kudu/127.0.201.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab.
I20260619 12:13:45.312085 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:41597
--webserver_interface=127.0.201.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab
--principal=kudu/127.0.201.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:44131,127.0.201.61:44453,127.0.201.60:41597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:13:45.336710 23063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } attempt: 3
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:45.344434 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.337214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34908 (local address 127.0.201.61:44453)
0619 12:13:45.337297 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.337299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.337310 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:13:45.337336 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.337337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.337347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.337409 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.337794 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.338276 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.339278 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.339490 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.339543 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.341648 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.341649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.341650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.341659 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.343166 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.343595 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.343596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.343597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.343660 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.343869 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.343870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.343871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.343992 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.344069 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.344224 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.344307 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:13:45.344463 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.337139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34908)
0619 12:13:45.337220 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.337231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.337276 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.337443 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.337444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.337633 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.337714 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.337716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.338353 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.338354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.339163 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.339166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.339561 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.339563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.339641 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.340101 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.340107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.341545 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.343299 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.343300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.343300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.343504 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.343728 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.343728 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.343729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.343791 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.344061 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.344062 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.344111 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.344200 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.344367 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 12:13:45.344789 23063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:45.345156 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.345064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37390)
0619 12:13:45.345106 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.345135 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 12:13:45.345456 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.345402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37398)
0619 12:13:45.345424 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.345444 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 12:13:45.345556 23063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41597: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
I20260619 12:13:45.355341 23138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } attempt: 1
I20260619 12:13:45.355829 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.355690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37410)
0619 12:13:45.355751 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.355793 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 12:13:45.356176 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.356073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37424)
0619 12:13:45.356121 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.356146 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:13:45.356259 23138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41597: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
I20260619 12:13:45.376693 23063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } attempt: 1
I20260619 12:13:45.377149 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.377016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37440)
0619 12:13:45.377078 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.377121 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:13:45.377550 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.377449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37456)
0619 12:13:45.377498 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.377531 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 12:13:45.377660 23063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41597: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
W20260619 12:13:45.398068 23145 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:13:45.398324 23145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:45.398355 23145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:45.398397 23145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:45.399600 23145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:45.399667 23145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:45.399693 23145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:45.399737 23145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:45.399770 23145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:45.401311 23145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:44131,127.0.201.61:44453,127.0.201.60:41597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab
--principal=kudu/127.0.201.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:41597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.201.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:45.401592 23145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:45.401826 23145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:45.406591 23150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:45.406615 23151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:45.406602 23153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:45.406821 23145 server_base.cc:1034] running on GCE node
I20260619 12:13:45.407042 23145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:45.407375 23145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:45.408550 23145 hybrid_clock.cc:648] HybridClock initialized: now 1781871225408538 us; error 29 us; skew 500 ppm
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:45.411056 23145 init.cc:377] Logged in from keytab as kudu/127.0.201.60@KRBTEST.COM (short username kudu)
I20260619 12:13:45.411648 23145 webserver.cc:460] Webserver started at http://127.0.201.60:43293/ using document root <none> and password file <none>
I20260619 12:13:45.411885 23145 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:45.411964 23145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:45.412123 23145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:45.413208 23145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data/instance:
uuid: "fa53a15c88b844f48bd67ca9df3020c8"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "d40f711ad1b8a9e6c73d5c0937b31d11"
server_key_iv: "be703babc22d38464c8b36f4237514e7"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.413583 23145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal/instance:
uuid: "fa53a15c88b844f48bd67ca9df3020c8"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "d40f711ad1b8a9e6c73d5c0937b31d11"
server_key_iv: "be703babc22d38464c8b36f4237514e7"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.414963 23145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 12:13:45.415895 23160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:45.416070 23145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:45.416174 23145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal
uuid: "fa53a15c88b844f48bd67ca9df3020c8"
format_stamp: "Formatted at 2026-06-19 12:13:45 on dist-test-slave-c910"
server_key: "d40f711ad1b8a9e6c73d5c0937b31d11"
server_key_iv: "be703babc22d38464c8b36f4237514e7"
server_key_version: "encryptionkey@0"
I20260619 12:13:45.416278 23145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:45.424393 23138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } attempt: 2
I20260619 12:13:45.424793 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.424661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37458)
0619 12:13:45.424718 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.424766 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:13:45.425091 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.425003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37460)
0619 12:13:45.425045 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.425068 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 12:13:45.425181 23138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41597: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41597: connect: Connection refused (error 111)
I20260619 12:13:45.444329 23145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:45.446362 23145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:45.446522 23145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:45.450723 23145 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.60:41597
I20260619 12:13:45.450757 23212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.60:41597 every 8 connection(s)
I20260619 12:13:45.451100 23145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
I20260619 12:13:45.452039 23213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:45.452824 23063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } attempt: 2
I20260619 12:13:45.453879 23213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:45.457274 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 23145
I20260619 12:13:45.457384 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781870963166527-804-0/minicluster-data/master-2/wal/instance
I20260619 12:13:45.457903 804 external_mini_cluster.cc:1428] Setting key fe255b30fb9283cced1776231d99373b
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:45.463104 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.454765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.0.201.62:44131)
0619 12:13:45.454816 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.454818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.454983 (+ 165us) server_negotiation.cc:407] Connection header received
0619 12:13:45.455058 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.455059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.455068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.455137 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.455642 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.456360 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.457454 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.457657 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.457736 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.460027 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.460028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.460028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.460036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.461656 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.462071 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.462073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.462073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.462127 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.462399 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.462400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.462400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.462609 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.462691 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.462935 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.463012 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:13:45.463294 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.453212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37462)
0619 12:13:45.453302 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.453315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.453371 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.454248 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.454249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.454421 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.454474 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.454477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.455775 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.455776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.456528 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.456529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.457240 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.457241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.457307 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.457753 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.457759 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.459812 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.461955 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.461956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.461957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.462186 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.462432 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.462432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.462433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.462482 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.462942 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.462943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.462988 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.463038 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.463153 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 12:13:45.463362 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.454742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:44131 (local address 127.0.0.1:36180)
0619 12:13:45.454894 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.454925 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.455000 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.455157 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.455159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.455466 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.455550 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.455554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.456449 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.456450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.457328 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.457331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.457777 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.457778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.457848 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.458355 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.458366 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.459940 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.461720 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.461725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.461727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.462002 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.462202 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.462203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.462203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.462321 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.462682 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.462688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.462812 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.462916 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.463065 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260619 12:13:45.463726 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.453444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.0.201.60:41597)
0619 12:13:45.453621 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.453629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.453653 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:13:45.454063 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.454068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.454114 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.454236 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.454590 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.455704 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.456576 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.457148 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.457256 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.459895 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.459903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.459903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.459910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.461878 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.462283 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.462286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.462287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.462345 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.462554 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.462554 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.462555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.462841 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.463476 (+ 635us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.463522 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.463595 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260619 12:13:45.463853 23213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:13:45.466032 23063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10: Bootstrap starting.
I20260619 12:13:45.466830 23063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:45.467159 23063 log.cc:824] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:45.468240 23063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10: No bootstrap required, opened a new log
I20260619 12:13:45.468586 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.458502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.0.201.60:41597)
0619 12:13:45.458662 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.458664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.458803 (+ 139us) server_negotiation.cc:407] Connection header received
0619 12:13:45.458875 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.458876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.458885 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.458946 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.459633 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.460260 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.461261 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.461552 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.461628 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.464758 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.464759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.464760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.464766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.467064 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.467520 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.467540 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.467541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.467613 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.467876 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.467877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.467877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.468007 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.468147 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.468372 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.468449 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:45.470402 23063 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.470549 23063 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7c246155344cb1833349b33eefcc10, State: Initialized, Role: FOLLOWER
I20260619 12:13:45.470649 23063 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.470969 23063 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:45.470934 23220 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.471045 23220 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:45.472137 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.464199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.0.201.61:44453)
0619 12:13:45.464251 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.464253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.464261 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:45.464293 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.464294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.464307 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.464361 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.464674 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.465327 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.466011 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.466347 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.466412 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.469239 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.469241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.469241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.469248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.470885 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.471294 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.471295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.471296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.471353 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.471611 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.471612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.471612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.471743 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.471812 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.471971 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.472045 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:13:45.472177 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.464072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:44453 (local address 127.0.0.1:34920)
0619 12:13:45.464163 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.464172 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.464206 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.464385 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.464386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.464530 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.464585 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.464587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.465423 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.465424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.465927 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.465929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.467157 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.467158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.467276 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.467645 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.467650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.469142 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.470980 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.470981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.470981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.471230 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.471430 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.471431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.471431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.471508 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.471807 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.471808 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.471858 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.471934 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.472029 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
W20260619 12:13:45.472869 23231 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:45.472926 23231 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:45.472965 23213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:45.473747 22995 master_runner.cc:186] Error getting master registration for 127.0.201.61:44453: OK, instance_id { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" instance_seqno: 1781871225274170 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:13:45.476662 22995 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:13:45.478780 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.469140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36184 (local address 127.0.201.62:44131)
0619 12:13:45.469177 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.469178 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.469212 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:13:45.469232 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.469232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.469242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.469301 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.470059 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.471233 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.472255 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.472529 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.472593 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.475370 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.475372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.475372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.475379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.477272 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.477685 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.477686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.477686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.477741 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.477939 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.477939 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.477940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.478063 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.478186 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.478599 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.478684 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:13:45.481642 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.473209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37484)
0619 12:13:45.473254 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.473265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.473301 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.473492 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.473492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.473648 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.473683 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.473685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.474524 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.474525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.475355 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.475357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.475841 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.475842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.475887 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.476234 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.476239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.478047 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.479868 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.479868 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.479869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.480089 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.480870 (+ 781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.480870 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.480871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.480938 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.481284 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.481285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.481333 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.481412 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.481520 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:13:45.481722 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.473329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.0.201.60:41597)
0619 12:13:45.473367 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.473369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.473376 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:13:45.473395 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.473395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.473406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.473456 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.473757 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.474437 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.475438 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.475740 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.475798 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.478130 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.478131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.478132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.478138 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.479780 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.480728 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.480729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.480729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.480789 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.481016 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.481017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.481017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.481205 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.481272 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.481449 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.481587 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":20}
I20260619 12:13:45.483381 23213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: Bootstrap starting.
I20260619 12:13:45.484165 23213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:45.484473 23213 log.cc:824] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:45.485337 23213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: No bootstrap required, opened a new log
I20260619 12:13:45.487725 23213 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.487877 23213 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa53a15c88b844f48bd67ca9df3020c8, State: Initialized, Role: FOLLOWER
I20260619 12:13:45.487958 23213 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.488296 23213 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:45.488255 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.488454 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:45.488626 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.479329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.0.201.61:44453)
0619 12:13:45.479383 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.479385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.479486 (+ 101us) server_negotiation.cc:407] Connection header received
0619 12:13:45.479532 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.479533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.479545 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.479605 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.480172 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.480626 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.481517 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.481820 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.481886 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.485082 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.485084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.485084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.485092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.486848 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.487293 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.487294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.487294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.487382 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.487644 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.487645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.487645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.487783 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.487940 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.488434 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.488517 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 12:13:45.490001 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 12:13:45.490641 23245 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:45.491046 23245 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:45.491065 23036 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ff7c246155344cb1833349b33eefcc10"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36184
I20260619 12:13:45.491221 23036 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 12:13:45.491274 23036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:45.492231 23036 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.492466 23036 leader_election.cc:288] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [CANDIDATE]: Term 1 election: Requested vote from peers 0f36d16f851846f48450e7ba49e59c59 (127.0.201.61:44453), fa53a15c88b844f48bd67ca9df3020c8 (127.0.201.60:41597)
I20260619 12:13:45.492966 23187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff7c246155344cb1833349b33eefcc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fa53a15c88b844f48bd67ca9df3020c8"
I20260619 12:13:45.493068 23187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:45.491559 23145 master_runner.cc:186] Error getting master registration for 127.0.201.61:44453: OK, instance_id { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" instance_seqno: 1781871225274170 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:13:45.493932 23145 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:13:45.493955 23187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff7c246155344cb1833349b33eefcc10 in term 1.
I20260619 12:13:45.494143 23012 leader_election.cc:302] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa53a15c88b844f48bd67ca9df3020c8, ff7c246155344cb1833349b33eefcc10; no voters:
I20260619 12:13:45.494266 23220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:45.494441 23220 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [term 1 LEADER]: Becoming Leader. State: Replica: ff7c246155344cb1833349b33eefcc10, State: Running, Role: LEADER
I20260619 12:13:45.494537 23220 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.495033 23251 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7c246155344cb1833349b33eefcc10. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.495249 23251 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:45.504007 23253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:45.504248 23253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:45.506255 23187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [term 1 FOLLOWER]: Refusing update from remote peer ff7c246155344cb1833349b33eefcc10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:45.506635 23251 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:45.507942 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7c246155344cb1833349b33eefcc10. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.508038 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:45.515421 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.503469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.0.201.61:44453)
0619 12:13:45.503517 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.503519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.505133 (+ 1614us) server_negotiation.cc:407] Connection header received
0619 12:13:45.505247 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.505249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.505262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.505293 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.506132 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.506835 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.507834 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.508607 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.508654 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.511532 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.511534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.511534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.511542 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.513626 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.514163 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.514164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.514165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.514225 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.514514 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.514515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.514516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.514665 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.514741 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.515248 (+ 507us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.515322 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:13:45.516261 23138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } attempt: 3
I20260619 12:13:45.516682 23251 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.516777 23251 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:45.520118 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.520217 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 12:13:45 dist-test-slave-c910 krb5kdc[22982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:13:45.523594 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.503898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.0.201.60:41597)
0619 12:13:45.503940 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.503942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.509214 (+ 5272us) server_negotiation.cc:407] Connection header received
0619 12:13:45.509311 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.509312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.509323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.509352 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.510087 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.510852 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.511852 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.512174 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.512220 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.519109 (+ 6889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.519111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.519111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.519119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.521061 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.522476 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.522477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.522478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.522550 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.522851 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.522852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.522853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.522992 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.523089 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.523412 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.523496 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:13:45.526307 23257 mvcc.cc:205] Tried to move back new op lower bound from 7298544539670446080 to 7298544539626659840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544539670446080}]
I20260619 12:13:45.526791 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.516735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41597 (local address 127.0.0.1:37500)
0619 12:13:45.516783 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:45.516793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:45.516829 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:45.517196 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:45.517197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:45.517355 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:45.517395 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.517397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.518370 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.518371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.519208 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:45.519211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.520425 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.520426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:45.520486 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.520902 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:45.520908 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:45.523871 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:45.525669 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.525670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.525671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.525907 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.526124 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:45.526125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:45.526125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:45.526187 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:45.526458 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:45.526459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:45.526507 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 12:13:45.526578 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:13:45.526690 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:13:45.527144 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.516912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.0.201.60:41597)
0619 12:13:45.516956 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.516958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.516967 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:45.516986 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.516987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.516996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.517051 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.517545 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.518279 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.519378 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.519706 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.519768 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.523963 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.523964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.523965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.523971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.525579 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.525974 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.525975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.525975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.526026 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.526257 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.526258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.526258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.526395 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.526465 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.526971 (+ 506us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.527052 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 12:13:45.528716 23138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: Bootstrap starting.
I20260619 12:13:45.529464 23138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:45.529765 23138 log.cc:824] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:45.530037 23258 mvcc.cc:205] Tried to move back new op lower bound from 7298544539670446080 to 7298544539626659840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544539670446080}]
I20260619 12:13:45.530866 23138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: No bootstrap required, opened a new log
I20260619 12:13:45.531291 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:45.503577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.0.201.62:44131)
0619 12:13:45.503628 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.503630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.509280 (+ 5650us) server_negotiation.cc:407] Connection header received
0619 12:13:45.509304 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.509305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.509318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.509369 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.513355 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.514084 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.515613 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:45.515891 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:45.515934 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:45.521867 (+ 5933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:45.521868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:45.521869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:45.521876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:45.523501 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.530199 (+ 6698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.530201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.530201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.530275 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:45.530495 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:45.530496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:45.530496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:45.530630 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:45.530743 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:45.531078 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 12:13:45.531172 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 12:13:45.532943 23253 catalog_manager.cc:1347] Generated new cluster ID: bfe24dd399ab4463b1e0b615783113bf
I20260619 12:13:45.532989 23253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:45.533011 23138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.533152 23138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f36d16f851846f48450e7ba49e59c59, State: Initialized, Role: FOLLOWER
I20260619 12:13:45.533228 23138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } }
I20260619 12:13:45.533548 23138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:45.535363 23112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff7c246155344cb1833349b33eefcc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0f36d16f851846f48450e7ba49e59c59"
I20260619 12:13:45.534880 23110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:45.536664 23110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [term 1 FOLLOWER]: Refusing update from remote peer ff7c246155344cb1833349b33eefcc10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:45.537009 23220 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:45.539737 23278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.539893 23278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:45.539979 23278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7c246155344cb1833349b33eefcc10. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.540076 23278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:45.540150 23278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.540241 23278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:45.541062 23112 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ff7c246155344cb1833349b33eefcc10 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260619 12:13:45.542181 23276 mvcc.cc:205] Tried to move back new op lower bound from 7298544539670446080 to 7298544539626659840. Current Snapshot: MvccSnapshot[applied={T|T < 7298544539670446080}]
W20260619 12:13:45.543955 23274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:45.544010 23274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:45.547008 23220 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff7c246155344cb1833349b33eefcc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff7c246155344cb1833349b33eefcc10" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 44131 } } peers { permanent_uuid: "0f36d16f851846f48450e7ba49e59c59" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 44453 } } peers { permanent_uuid: "fa53a15c88b844f48bd67ca9df3020c8" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41597 } } }
I20260619 12:13:45.547120 23220 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:45.549758 23253 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:45.550473 23253 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:45.578037 23253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff7c246155344cb1833349b33eefcc10: Generated new TSK 0
I20260619 12:13:45.578249 23253 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:13:45.602487 23066 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.600782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36214 (local address 127.0.201.62:44131)
0619 12:13:45.600829 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.600832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.601798 (+ 966us) server_negotiation.cc:407] Connection header received
0619 12:13:45.601912 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.601913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.601925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.601956 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.602411 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36214: BlockingRecv error: recv got EOF from 127.0.0.1:36214 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 12:13:45.603312 23215 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.601164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.0.201.60:41597)
0619 12:13:45.601204 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.601206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.602747 (+ 1541us) server_negotiation.cc:407] Connection header received
0619 12:13:45.602845 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.602846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.602857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.602887 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.603252 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37514: BlockingRecv error: recv got EOF from 127.0.0.1:37514 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 12:13:45.604070 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.601499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.0.201.61:44453)
0619 12:13:45.601541 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.601542 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.603540 (+ 1998us) server_negotiation.cc:407] Connection header received
0619 12:13:45.603640 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.603641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.603652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.603694 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.604009 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34954: BlockingRecv error: recv got EOF from 127.0.0.1:34954 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 12:13:45.606135 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.605393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.0.201.61:44453)
0619 12:13:45.605428 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.605430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.605610 (+ 180us) server_negotiation.cc:407] Connection header received
0619 12:13:45.605733 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.605734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.605753 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.605784 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.606089 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34962: BlockingRecv error: recv got EOF from 127.0.0.1:34962 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 12:13:45.609120 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.608354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.0.201.61:44453)
0619 12:13:45.608387 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.608389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.608600 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:13:45.608714 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.608715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.608725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.608754 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.609074 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34970: BlockingRecv error: recv got EOF from 127.0.0.1:34970 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 12:13:45.613117 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.612404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.0.201.61:44453)
0619 12:13:45.612437 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.612438 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.612604 (+ 166us) server_negotiation.cc:407] Connection header received
0619 12:13:45.612714 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.612715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.612726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.612755 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.613073 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34974: BlockingRecv error: recv got EOF from 127.0.0.1:34974 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 12:13:45.619321 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.618337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.0.201.61:44453)
0619 12:13:45.618376 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.618378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.618678 (+ 300us) server_negotiation.cc:407] Connection header received
0619 12:13:45.618819 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.618820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.618831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.618861 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.619263 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34976: BlockingRecv error: recv got EOF from 127.0.0.1:34976 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 12:13:45.629623 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.628758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.0.201.61:44453)
0619 12:13:45.628799 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.628801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.629045 (+ 244us) server_negotiation.cc:407] Connection header received
0619 12:13:45.629149 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.629150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.629161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.629193 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.629562 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34984: BlockingRecv error: recv got EOF from 127.0.0.1:34984 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 12:13:45.647886 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.646987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34992 (local address 127.0.201.61:44453)
0619 12:13:45.647030 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.647032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.647295 (+ 263us) server_negotiation.cc:407] Connection header received
0619 12:13:45.647393 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.647394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.647406 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.647435 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.647824 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34992: BlockingRecv error: recv got EOF from 127.0.0.1:34992 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 12:13:45.682142 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.681337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.0.201.61:44453)
0619 12:13:45.681388 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.681390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.681532 (+ 142us) server_negotiation.cc:407] Connection header received
0619 12:13:45.681595 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.681596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.681608 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.681689 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.682054 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34998: BlockingRecv error: recv got EOF from 127.0.0.1:34998 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 12:13:45.748224 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.747321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.0.201.61:44453)
0619 12:13:45.747430 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.747432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.747571 (+ 139us) server_negotiation.cc:407] Connection header received
0619 12:13:45.747696 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.747697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.747710 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.747766 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.748149 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35002: BlockingRecv error: recv got EOF from 127.0.0.1:35002 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 12:13:45.878458 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:45.877647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.0.201.61:44453)
0619 12:13:45.877723 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:45.877725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:45.877813 (+ 88us) server_negotiation.cc:407] Connection header received
0619 12:13:45.877913 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:45.877914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:45.877928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:45.877988 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:45.878383 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35014: BlockingRecv error: recv got EOF from 127.0.0.1:35014 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 12:13:46.136468 23142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:46.135624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35030 (local address 127.0.201.61:44453)
0619 12:13:46.135700 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:46.135702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:46.135846 (+ 144us) server_negotiation.cc:407] Connection header received
0619 12:13:46.135932 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:46.135934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:46.135951 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:46.136019 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:46.136398 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35030: BlockingRecv error: recv got EOF from 127.0.0.1:35030 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:13:46.491603 23245 catalog_manager.cc:1259] Loaded cluster ID: bfe24dd399ab4463b1e0b615783113bf
I20260619 12:13:46.491685 23245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: loading cluster ID for follower catalog manager: success
I20260619 12:13:46.492823 23245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: acquiring CA information for follower catalog manager: success
I20260619 12:13:46.493084 23245 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa53a15c88b844f48bd67ca9df3020c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:13:46.544473 23274 catalog_manager.cc:1259] Loaded cluster ID: bfe24dd399ab4463b1e0b615783113bf
I20260619 12:13:46.544595 23274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: loading cluster ID for follower catalog manager: success
I20260619 12:13:46.545456 23274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: acquiring CA information for follower catalog manager: success
I20260619 12:13:46.545790 23274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0f36d16f851846f48450e7ba49e59c59: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:13:46.652527 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:46.649663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.0.201.61:44453)
0619 12:13:46.649790 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:46.649793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:46.649866 (+ 73us) server_negotiation.cc:407] Connection header received
0619 12:13:46.649964 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:46.649965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:46.650036 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:46.650125 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:46.650301 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.650809 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.651678 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.651882 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.651954 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:46.652129 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:46.652250 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:46.652351 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:46.652396 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 12:13:46.652418 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260619 12:13:46.656843 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:46.654410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.0.201.60:41597)
0619 12:13:46.654512 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:46.654514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:46.654522 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:46.654554 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:46.654555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:46.654590 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:46.654665 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:46.654763 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.655311 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.656003 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.656204 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.656262 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:46.656385 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:46.656485 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:46.656543 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:46.656689 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 12:13:46.656726 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":56,"threads_started":1}
I20260619 12:13:46.661422 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:46.658480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.0.201.61:44453)
0619 12:13:46.658534 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:46.658536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:46.658662 (+ 126us) server_negotiation.cc:407] Connection header received
0619 12:13:46.658768 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:46.658768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:46.658778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:46.658807 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:46.659050 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.659648 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.660645 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.660949 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.661015 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:46.661166 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:46.661203 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:46.661253 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:46.661323 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 12:13:46.661349 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:46.661458 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:46.658853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37522 (local address 127.0.201.60:41597)
0619 12:13:46.658894 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:46.658896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:46.658920 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:13:46.659031 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:46.659032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:46.659057 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:46.659134 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:46.659269 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.659957 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.660708 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.661009 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.661068 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:46.661196 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:46.661227 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:46.661276 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:46.661371 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 12:13:46.661391 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 12:13:46.661917 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:46.658551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.0.201.62:44131)
0619 12:13:46.658667 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:46.658669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:46.658678 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:46.658746 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:46.658747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:46.658837 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:46.658963 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:46.659107 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.659926 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.660911 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:46.661279 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:46.661509 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:46.661584 (+ 75us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:46.661683 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:46.661763 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:46.661814 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 12:13:46.661836 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260619 12:13:46.663671 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 22995
I20260619 12:13:46.669404 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 23070
I20260619 12:13:46.674347 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 23145
2026-06-19T12:13:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2837 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:13:46 dist-test-slave-c910 krb5kdc[23363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:13:46 dist-test-slave-c910 krb5kdc[23363](info): set up 2 sockets
Jun 19 12:13:46 dist-test-slave-c910 krb5kdc[23363](info): commencing operation
krb5kdc: starting...
W20260619 12:13:47.719475 804 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/test-user.keytab.
Jun 19 12:13:47 dist-test-slave-c910 krb5kdc[23363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:13:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.201.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
Entry for principal kudu/127.0.201.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab.
I20260619 12:13:47.834657 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42293
--webserver_interface=127.0.201.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42293,127.0.201.61:43907,127.0.201.60:41879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:13:47.914889 23376 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:13:47.915115 23376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:47.915132 23376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:47.915184 23376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:47.916344 23376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:47.916386 23376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:47.916400 23376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:47.916441 23376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:47.916472 23376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:47.917933 23376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42293,127.0.201.61:43907,127.0.201.60:41879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.62.keytab
--principal=kudu/127.0.201.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.62:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.201.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:47.918196 23376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:47.918431 23376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:47.923319 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:47.923373 23381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:47.923319 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:47.923553 23376 server_base.cc:1034] running on GCE node
I20260619 12:13:47.923827 23376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:47.924121 23376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:47.925282 23376 hybrid_clock.cc:648] HybridClock initialized: now 1781871227925271 us; error 33 us; skew 500 ppm
Jun 19 12:13:47 dist-test-slave-c910 krb5kdc[23363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:47.932483 23376 init.cc:377] Logged in from keytab as kudu/127.0.201.62@KRBTEST.COM (short username kudu)
I20260619 12:13:47.933022 23376 webserver.cc:460] Webserver started at http://127.0.201.62:37421/ using document root <none> and password file <none>
I20260619 12:13:47.933252 23376 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:47.933320 23376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:47.933471 23376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:47.934553 23376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data/instance:
uuid: "b700c9d70f074661b42ce628e89a9724"
format_stamp: "Formatted at 2026-06-19 12:13:47 on dist-test-slave-c910"
server_key: "4873d1df49616baad2d7289dc09a2423"
server_key_iv: "b0863cd567264951918048bc7e23a03d"
server_key_version: "encryptionkey@0"
I20260619 12:13:47.934924 23376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal/instance:
uuid: "b700c9d70f074661b42ce628e89a9724"
format_stamp: "Formatted at 2026-06-19 12:13:47 on dist-test-slave-c910"
server_key: "4873d1df49616baad2d7289dc09a2423"
server_key_iv: "b0863cd567264951918048bc7e23a03d"
server_key_version: "encryptionkey@0"
I20260619 12:13:47.936311 23376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 12:13:47.937073 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:47.937287 23376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 12:13:47.937395 23376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal
uuid: "b700c9d70f074661b42ce628e89a9724"
format_stamp: "Formatted at 2026-06-19 12:13:47 on dist-test-slave-c910"
server_key: "4873d1df49616baad2d7289dc09a2423"
server_key_iv: "b0863cd567264951918048bc7e23a03d"
server_key_version: "encryptionkey@0"
I20260619 12:13:47.937484 23376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:47.960790 23376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:47.963214 23376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:47.963388 23376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:47.967449 23376 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.62:42293
I20260619 12:13:47.967482 23443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.62:42293 every 8 connection(s)
I20260619 12:13:47.967839 23376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/data/info.pb
I20260619 12:13:47.968799 23444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:47.969432 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 23376
I20260619 12:13:47.969535 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-0/wal/instance
I20260619 12:13:47.969722 804 external_mini_cluster.cc:1428] Setting key 6259fbf5634b4180f8fd02b7eab00e09
I20260619 12:13:47.971170 23444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.201.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.61@KRBTEST.COM" created.
Jun 19 12:13:47 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:47.983990 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:47.972192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.0.201.62:42293)
0619 12:13:47.972334 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:47.972340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:47.972371 (+ 31us) server_negotiation.cc:407] Connection header received
0619 12:13:47.972850 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:47.972852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:47.972871 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:47.972983 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:47.973581 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:47.974421 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:47.975348 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:47.975659 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:47.975799 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:47.980524 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:47.980534 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:47.980534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:47.980542 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:47.982252 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:47.982765 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:47.982768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:47.982768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:47.982843 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:47.983118 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:47.983120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:47.983120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:47.983402 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:47.983467 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:47.983716 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 12:13:47.983823 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260619 12:13:47.984072 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:47.971989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42293 (local address 127.0.0.1:48796)
0619 12:13:47.972158 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:47.972192 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:47.972691 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:47.972990 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:47.972993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:47.973337 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:47.973485 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:47.973490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:47.974532 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:47.974532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:47.975258 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:47.975261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:47.975774 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:47.975775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:47.975837 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:47.976420 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:47.976429 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:47.980428 (+ 3999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:47.982343 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:47.982350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:47.982353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:47.982639 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:47.982914 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:47.982915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:47.982915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:47.983037 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:47.983475 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:47.983481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:47.983567 (+ 86us) client_negotiation.cc:769] Sending connection context
0619 12:13:47.983690 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:13:47.983905 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":37,"thread_start_us":87,"threads_started":1}
I20260619 12:13:47.984833 23444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:47.985222 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:47.985080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57486)
0619 12:13:47.985138 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:47.985191 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 12:13:47.985376 23394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
I20260619 12:13:47.985831 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:47.985731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57490)
0619 12:13:47.985783 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:47.985813 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:13:47.985946 23444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:43907: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab.
Entry for principal kudu/127.0.201.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab.
I20260619 12:13:47.994295 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:43907
--webserver_interface=127.0.201.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab
--principal=kudu/127.0.201.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42293,127.0.201.61:43907,127.0.201.60:41879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:13:48.035228 23444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } attempt: 1
I20260619 12:13:48.035760 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.035608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57500)
0619 12:13:48.035683 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.035728 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 12:13:48.036111 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.036011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57504)
0619 12:13:48.036056 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.036080 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 12:13:48.036198 23444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:43907: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
W20260619 12:13:48.074553 23451 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:13:48.074772 23451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:48.074790 23451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:48.074841 23451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:48.075976 23451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:48.076017 23451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:48.076031 23451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:48.076082 23451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:48.076109 23451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:48.077564 23451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42293,127.0.201.61:43907,127.0.201.60:41879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.61.keytab
--principal=kudu/127.0.201.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.61:43907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.201.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:48.077813 23451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:48.078039 23451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:13:48.082860 23459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:48.082871 23456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:48.082880 23457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:48.082886 23451 server_base.cc:1034] running on GCE node
I20260619 12:13:48.083384 23451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:48.083750 23451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:48.084932 23451 hybrid_clock.cc:648] HybridClock initialized: now 1781871228084903 us; error 41 us; skew 500 ppm
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:48.087437 23451 init.cc:377] Logged in from keytab as kudu/127.0.201.61@KRBTEST.COM (short username kudu)
I20260619 12:13:48.087940 23451 webserver.cc:460] Webserver started at http://127.0.201.61:41091/ using document root <none> and password file <none>
I20260619 12:13:48.088130 23451 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:48.088171 23451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:48.088259 23451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:48.089329 23451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data/instance:
uuid: "4734fd64c3584548a01a5264df8059e5"
format_stamp: "Formatted at 2026-06-19 12:13:48 on dist-test-slave-c910"
server_key: "04d4db2ee49acf874d89df3872f66a68"
server_key_iv: "9495f1ea5e9369ac39ce636efd35e075"
server_key_version: "encryptionkey@0"
I20260619 12:13:48.089687 23451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal/instance:
uuid: "4734fd64c3584548a01a5264df8059e5"
format_stamp: "Formatted at 2026-06-19 12:13:48 on dist-test-slave-c910"
server_key: "04d4db2ee49acf874d89df3872f66a68"
server_key_iv: "9495f1ea5e9369ac39ce636efd35e075"
server_key_version: "encryptionkey@0"
I20260619 12:13:48.091003 23451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 12:13:48.091811 23466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:48.092000 23451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 12:13:48.092078 23451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal
uuid: "4734fd64c3584548a01a5264df8059e5"
format_stamp: "Formatted at 2026-06-19 12:13:48 on dist-test-slave-c910"
server_key: "04d4db2ee49acf874d89df3872f66a68"
server_key_iv: "9495f1ea5e9369ac39ce636efd35e075"
server_key_version: "encryptionkey@0"
I20260619 12:13:48.092130 23451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:48.104346 23444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } attempt: 2
I20260619 12:13:48.104846 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.104693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57518)
0619 12:13:48.104760 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.104806 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 12:13:48.105227 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.105134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57524)
0619 12:13:48.105179 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.105206 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:13:48.105365 23444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.61:43907: Network error: Client connection negotiation failed: client connection to 127.0.201.61:43907: connect: Connection refused (error 111)
I20260619 12:13:48.110219 23451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:48.112679 23451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:48.112843 23451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:48.116688 23451 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.61:43907
I20260619 12:13:48.116789 23518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.61:43907 every 8 connection(s)
I20260619 12:13:48.117117 23451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/data/info.pb
I20260619 12:13:48.118140 23519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:48.118832 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 23451
I20260619 12:13:48.118958 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-1/wal/instance
I20260619 12:13:48.119159 804 external_mini_cluster.cc:1428] Setting key 2efef104ceb0e5ad67a3f51258dc4042
I20260619 12:13:48.119820 23519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.201.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.201.60@KRBTEST.COM" created.
I20260619 12:13:48.129319 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.120550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.0.201.62:42293)
0619 12:13:48.120608 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.120610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.120821 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:13:48.121323 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.121325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.121349 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.121403 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.122046 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.122920 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.123969 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.124325 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.124403 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.126480 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.126481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.126482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.126489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.127915 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.128354 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.128355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.128355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.128427 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.128646 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.128647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.128647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.128761 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.128875 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.129186 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.129255 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:13:48.129627 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.120525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42293 (local address 127.0.0.1:48798)
0619 12:13:48.120703 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.120739 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.121211 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.121511 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.121521 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.121765 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.121952 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.121955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.123022 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.123023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.123883 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.123885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.124420 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.124421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.124523 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.125032 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.125040 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.126383 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.128002 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.128008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.128010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.128261 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.128487 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.128488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.128488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.128589 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.128833 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.128838 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.129006 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.129168 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.129350 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260619 12:13:48.130203 23519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:48.137914 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.130465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57526)
0619 12:13:48.130503 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.130514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.130558 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.130874 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.130875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.131034 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.131095 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.131096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.131976 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.131977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.132875 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.132878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.133346 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.133347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.133430 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.133802 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.133808 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.135088 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.136833 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.136834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.136835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.136991 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.137202 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.137203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.137203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.137256 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.137591 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.137592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.137629 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.137700 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.137803 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 12:13:48.137991 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.130541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.0.201.61:43907)
0619 12:13:48.130737 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.130741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.130757 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:13:48.130774 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.130776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.130787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.130848 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.131169 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.131885 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.132951 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.133245 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.133330 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.135186 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.135211 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.135212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.135218 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.136713 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.137080 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.137083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.137084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.137137 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.137339 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.137340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.137340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.137523 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.137614 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.137731 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.137870 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
I20260619 12:13:48.138895 23519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:48.139276 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.139134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48002)
0619 12:13:48.139182 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.139244 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:13:48.139400 23467 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
I20260619 12:13:48.139811 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.139717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48012)
0619 12:13:48.139767 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.139790 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 12:13:48.139914 23519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41879: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.201.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab.
Entry for principal kudu/127.0.201.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab.
I20260619 12:13:48.145776 804 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq9ki8M/build/release/bin/kudu
/tmp/dist-test-taskq9ki8M/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:41879
--webserver_interface=127.0.201.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab
--principal=kudu/127.0.201.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.201.20:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.201.62:42293,127.0.201.61:43907,127.0.201.60:41879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:13:48.189148 23519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } attempt: 1
I20260619 12:13:48.189702 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.189571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48024)
0619 12:13:48.189629 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.189670 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 12:13:48.190059 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.189960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48034)
0619 12:13:48.190012 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.190037 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:13:48.190143 23519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41879: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
I20260619 12:13:48.196527 23444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } attempt: 3
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:48.203473 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.196842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57528)
0619 12:13:48.196898 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.196917 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.196984 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.197139 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.197140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.197308 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.197372 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.197374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.198025 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.198026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.198554 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.198555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.198939 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.198940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.199019 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.199522 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.199526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.200754 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.202516 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.202517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.202517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.202676 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.202873 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.202873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.202873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.202924 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.203185 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.203185 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.203276 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.203329 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.203405 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 12:13:48.203545 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.196911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.0.201.61:43907)
0619 12:13:48.196971 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.196974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.196984 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:13:48.197052 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.197053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.197065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.197133 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.197453 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.197943 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.198637 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.198865 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.198941 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.200848 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.200849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.200849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.200857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.202421 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.202744 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.202745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.202746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.202807 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.202983 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.202984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.202984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.203126 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.203208 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.203373 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.203454 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:13:48.203866 23444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:48.204219 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.204119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48050)
0619 12:13:48.204174 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.204201 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:13:48.204551 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.204464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48060)
0619 12:13:48.204506 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.204535 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 12:13:48.204629 23444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41879: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
W20260619 12:13:48.228027 23526 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:13:48.228261 23526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:13:48.228291 23526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:13:48.228335 23526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:13:48.229444 23526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:13:48.229509 23526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:13:48.229534 23526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:13:48.229575 23526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:13:48.229600 23526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:13:48.231071 23526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.201.20:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.201.62:42293,127.0.201.61:43907,127.0.201.60:41879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/krb5kdc/kudu_127.0.201.60.keytab
--principal=kudu/127.0.201.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.201.60:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.201.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 10:56:02 UTC on ccycloud.quasar-wyewmr.root.comops.site
build id 4455436
I20260619 12:13:48.231376 23526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:13:48.231619 23526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:13:48.235795 23444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } attempt: 1
I20260619 12:13:48.236315 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.236157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48072)
0619 12:13:48.236213 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.236252 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 12:13:48.236352 23531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:48.236352 23534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:13:48.236380 23532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:13:48.236692 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.236571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48074)
0619 12:13:48.236634 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.236665 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 12:13:48.236698 23526 server_base.cc:1034] running on GCE node
W20260619 12:13:48.236774 23444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41879: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
I20260619 12:13:48.236928 23526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:13:48.237210 23526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:13:48.238350 23526 hybrid_clock.cc:648] HybridClock initialized: now 1781871228238327 us; error 34 us; skew 500 ppm
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:13:48.240743 23526 init.cc:377] Logged in from keytab as kudu/127.0.201.60@KRBTEST.COM (short username kudu)
I20260619 12:13:48.241250 23526 webserver.cc:460] Webserver started at http://127.0.201.60:35403/ using document root <none> and password file <none>
I20260619 12:13:48.241496 23526 fs_manager.cc:359] Metadata directory not provided
I20260619 12:13:48.241541 23526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:13:48.241680 23526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:13:48.242751 23526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data/instance:
uuid: "a6bc036599b24ed2b25d73dbfef7be36"
format_stamp: "Formatted at 2026-06-19 12:13:48 on dist-test-slave-c910"
server_key: "92f328f1b8e3d3cfc19d8b17d8a2d662"
server_key_iv: "04ded034db661827e30c244b7c02b15d"
server_key_version: "encryptionkey@0"
I20260619 12:13:48.243116 23526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal/instance:
uuid: "a6bc036599b24ed2b25d73dbfef7be36"
format_stamp: "Formatted at 2026-06-19 12:13:48 on dist-test-slave-c910"
server_key: "92f328f1b8e3d3cfc19d8b17d8a2d662"
server_key_iv: "04ded034db661827e30c244b7c02b15d"
server_key_version: "encryptionkey@0"
I20260619 12:13:48.244444 23526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:13:48.245187 23541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:48.245378 23526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 12:13:48.245464 23526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data,/tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal
uuid: "a6bc036599b24ed2b25d73dbfef7be36"
format_stamp: "Formatted at 2026-06-19 12:13:48 on dist-test-slave-c910"
server_key: "92f328f1b8e3d3cfc19d8b17d8a2d662"
server_key_iv: "04ded034db661827e30c244b7c02b15d"
server_key_version: "encryptionkey@0"
I20260619 12:13:48.245532 23526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:13:48.256203 23526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:13:48.258288 23519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } attempt: 2
I20260619 12:13:48.258457 23526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:13:48.258621 23526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:13:48.258761 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.258615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48086)
0619 12:13:48.258686 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.258724 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 12:13:48.259181 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.259068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48090)
0619 12:13:48.259120 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.259151 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:13:48.259300 23519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.201.60:41879: Network error: Client connection negotiation failed: client connection to 127.0.201.60:41879: connect: Connection refused (error 111)
I20260619 12:13:48.262712 23526 rpc_server.cc:307] RPC server started. Bound to: 127.0.201.60:41879
I20260619 12:13:48.262777 23593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.201.60:41879 every 8 connection(s)
I20260619 12:13:48.263104 23526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/data/info.pb
I20260619 12:13:48.264184 23594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:13:48.265699 23594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:13:48.270565 804 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq9ki8M/build/release/bin/kudu as pid 23526
I20260619 12:13:48.270648 804 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq9ki8M/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781870963166527-804-0/minicluster-data/master-2/wal/instance
I20260619 12:13:48.270867 804 external_mini_cluster.cc:1428] Setting key b8d902db92c9f9e5ebb7a13df288fc48
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
I20260619 12:13:48.274875 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.266393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.0.201.62:42293)
0619 12:13:48.266457 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.266460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.266644 (+ 184us) server_negotiation.cc:407] Connection header received
0619 12:13:48.267141 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.267142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.267154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.267247 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.267838 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.268311 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.269301 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.269515 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.269604 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.271896 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.271898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.271898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.271906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.273451 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.273932 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.273933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.273933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.273987 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.274251 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.274252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.274252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.274354 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.274403 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.274703 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.274784 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:13:48.274935 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.266341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.62:42293 (local address 127.0.0.1:48814)
0619 12:13:48.266514 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.266548 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.267023 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.267302 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.267310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.267547 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.267737 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.267741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.268411 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.268412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.269200 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.269202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.269634 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.269635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.269754 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.270201 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.270216 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.271807 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.273535 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.273541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.273542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.273857 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.274051 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.274052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.274052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.274187 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.274417 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.274422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.274580 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.274661 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.274779 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260619 12:13:48.276002 23594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:48.280562 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.271319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.0.201.60:41879)
0619 12:13:48.271492 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.271498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.271551 (+ 53us) server_negotiation.cc:407] Connection header received
0619 12:13:48.271623 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.271625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.271647 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.271719 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.272369 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.273501 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.274518 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.275020 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.275111 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.277666 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.277672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.277672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.277678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.279440 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.279777 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.279780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.279781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.279834 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.280048 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.280048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.280049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.280205 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.280367 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.280416 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.280491 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260619 12:13:48.282963 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.276352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57530 (local address 127.0.201.61:43907)
0619 12:13:48.276401 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.276403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.276411 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:48.276434 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.276436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.276445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.276508 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.276780 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.277224 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.278074 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.278381 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.278452 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.280181 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.280183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.280183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.280189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.282012 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.282305 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.282306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.282307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.282362 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.282526 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.282526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.282527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.282643 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.282707 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.282847 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.282903 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 12:13:48.282994 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.276266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.61:43907 (local address 127.0.0.1:57530)
0619 12:13:48.276313 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.276325 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.276362 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.276515 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.276516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.276632 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.276699 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.276701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.277298 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.277299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.277978 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.277981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.278485 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.278486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.278559 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.278946 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.278950 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.280103 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.282093 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.282094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.282094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.282241 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.282426 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.282426 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.282426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.282472 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.282708 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.282709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.282748 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.282825 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.282906 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:13:48.283411 23594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.62@KRBTEST.COM
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.60@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:13:48.289762 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.281395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.0.201.62:42293)
0619 12:13:48.281432 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.281434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.281442 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:48.281462 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.281463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.281472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.281528 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.281949 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.282636 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.283971 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.284212 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.284291 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.286975 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.286976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.286976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.286982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.288548 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.288956 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.288957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.288957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.289005 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.289214 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.289214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.289215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.289312 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.289418 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.289577 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.289654 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 12:13:48.290763 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.283870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48106 (local address 127.0.201.60:41879)
0619 12:13:48.283928 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.283929 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.283936 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:13:48.283972 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.283973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.283983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.284040 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.284301 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.284759 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.285731 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.285907 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.285983 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.287858 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.287859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.287860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.287866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.289449 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.289837 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.289838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.289839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.289908 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.290110 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.290111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.290111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.290263 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.290321 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.290498 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.290645 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":42}
I20260619 12:13:48.290769 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.283797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48106)
0619 12:13:48.283867 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.283879 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.283923 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.284059 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.284059 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.284174 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.284205 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.284206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.284820 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.284821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.285647 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.285650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.285980 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.285981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.286027 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.286414 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.286418 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.287753 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.289514 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.289536 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.289537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.289760 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.289988 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.289988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.289989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.290055 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.290342 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.290343 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.290391 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.290473 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.290581 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 12:13:48.292484 23594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: Bootstrap starting.
I20260619 12:13:48.293557 23594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:48.293908 23594 log.cc:824] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:48.294781 23594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: No bootstrap required, opened a new log
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.201.61@KRBTEST.COM
I20260619 12:13:48.296607 23594 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.296763 23594 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bc036599b24ed2b25d73dbfef7be36, State: Initialized, Role: FOLLOWER
I20260619 12:13:48.296857 23594 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.297223 23594 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:48.297174 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.297350 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.298350 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.290410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.0.201.61:43907)
0619 12:13:48.290446 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.290448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.290455 (+ 7us) server_negotiation.cc:407] Connection header received
0619 12:13:48.290508 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.290509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.290518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.290564 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.291467 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.291902 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.292786 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.292963 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.293015 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.295772 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.295773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.295774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.295781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.297205 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.297601 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.297603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.297603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.297661 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.297860 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.297861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.297862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.297972 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.298137 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.298172 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.298253 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 12:13:48.299356 23611 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:48.299446 23611 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:48.299657 804 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:13:48.300449 23418 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b700c9d70f074661b42ce628e89a9724"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48822
I20260619 12:13:48.299650 23526 master_runner.cc:186] Error getting master registration for 127.0.201.62:42293: OK, instance_id { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" instance_seqno: 1781871227966321 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:13:48.302556 23526 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:13:48.311923 23444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } attempt: 2
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.62@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:13:48.320142 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.312287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48118)
0619 12:13:48.312356 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.312371 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.312434 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.312600 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.312601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.312802 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.312862 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.312864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.313751 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.313752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.314563 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.314566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.315068 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.315070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.315134 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.315622 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.315628 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.317183 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.318914 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.318916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.318916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.319150 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.319393 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.319393 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.319394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.319466 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.319793 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.319794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.319846 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.319925 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.320027 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 12:13:48.320459 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.312365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.0.201.60:41879)
0619 12:13:48.312422 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.312424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.312435 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:13:48.312517 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.312518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.312529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.312582 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.312943 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.313668 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.314655 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.314952 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.315052 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.317264 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.317265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.317265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.317272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.318828 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.319263 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.319264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.319265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.319322 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.319537 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.319537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.319538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.319718 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.319784 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.320283 (+ 499us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.320366 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 12:13:48.322218 23444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724: Bootstrap starting.
I20260619 12:13:48.322983 23444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:48.323357 23444 log.cc:824] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:48.324268 23444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724: No bootstrap required, opened a new log
I20260619 12:13:48.325695 23444 raft_consensus.cc:348] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.325850 23444 raft_consensus.cc:729] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700c9d70f074661b42ce628e89a9724, State: Initialized, Role: FOLLOWER
I20260619 12:13:48.325942 23444 consensus_queue.cc:260] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.326275 23444 sys_catalog.cc:564] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:48.326241 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.326305 23418 raft_consensus.cc:482] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 12:13:48.326336 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.326509 23418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:48.327381 23418 raft_consensus.cc:504] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.328383 23568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b700c9d70f074661b42ce628e89a9724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a6bc036599b24ed2b25d73dbfef7be36"
I20260619 12:13:48.328521 23568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [term 0 FOLLOWER]: Advancing to term 1
W20260619 12:13:48.329002 23625 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:13:48.329103 23625 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:48.329447 23568 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b700c9d70f074661b42ce628e89a9724 in term 1.
I20260619 12:13:48.329675 23392 leader_election.cc:302] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6bc036599b24ed2b25d73dbfef7be36, b700c9d70f074661b42ce628e89a9724; no voters:
I20260619 12:13:48.329794 23614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:13:48.329912 23614 raft_consensus.cc:686] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [term 1 LEADER]: Becoming Leader. State: Replica: b700c9d70f074661b42ce628e89a9724, State: Running, Role: LEADER
I20260619 12:13:48.330010 23614 consensus_queue.cc:237] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.330411 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b700c9d70f074661b42ce628e89a9724. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.330484 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:48.330859 23629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:13:48.331152 23629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:13:48.330055 23376 master_runner.cc:186] Error getting master registration for 127.0.201.61:43907: OK, instance_id { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" instance_seqno: 1781871228115631 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:13:48.332525 23376 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:13:48.332576 23418 leader_election.cc:288] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [CANDIDATE]: Term 1 election: Requested vote from peers 4734fd64c3584548a01a5264df8059e5 (127.0.201.61:43907), a6bc036599b24ed2b25d73dbfef7be36 (127.0.201.60:41879)
I20260619 12:13:48.334837 23568 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [term 1 FOLLOWER]: Refusing update from remote peer b700c9d70f074661b42ce628e89a9724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:48.335260 23627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:13:48.336611 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b700c9d70f074661b42ce628e89a9724. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.336730 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.344467 23627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.344575 23627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:48.345417 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.345506 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.345501 23629 catalog_manager.cc:1347] Generated new cluster ID: c34fd4f9f57e488b94a1c9c5feae0348
I20260619 12:13:48.345538 23629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:13:48.346493 23630 mvcc.cc:205] Tried to move back new op lower bound from 7298544551257395200 to 7298544551240429568. Current Snapshot: MvccSnapshot[applied={T|T < 7298544551257395200 or (T in {7298544551257395200})}]
I20260619 12:13:48.347860 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.333821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.0.201.60:41879)
0619 12:13:48.333858 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.333859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.335631 (+ 1772us) server_negotiation.cc:407] Connection header received
0619 12:13:48.335817 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.335818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.335828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.335857 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.337602 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.338334 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.340281 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.340941 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.341004 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.343603 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.343604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.343604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.343612 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.346319 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.347076 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.347078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.347078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.347128 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.347333 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.347333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.347334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.347452 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.347524 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.347704 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.347774 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 12:13:48.348840 23638 mvcc.cc:205] Tried to move back new op lower bound from 7298544551257395200 to 7298544551240429568. Current Snapshot: MvccSnapshot[applied={T|T < 7298544551257395200}]
I20260619 12:13:48.350418 23519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } attempt: 3
I20260619 12:13:48.355983 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.333991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.0.201.61:43907)
0619 12:13:48.334042 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.334044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.348206 (+ 14162us) server_negotiation.cc:407] Connection header received
0619 12:13:48.348311 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.348312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.348323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.348350 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.348955 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.349677 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.350899 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.351254 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.351305 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.352822 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.352824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.352824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.352830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.354469 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.354840 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.354841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.354842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.354894 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.355113 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.355114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.355114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.355262 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.355330 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.355546 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.355621 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 12:13:48 dist-test-slave-c910 krb5kdc[23363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.201.61@KRBTEST.COM for kudu/127.0.201.60@KRBTEST.COM
I20260619 12:13:48.360363 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.333747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48828 (local address 127.0.201.62:42293)
0619 12:13:48.333798 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.333801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.335717 (+ 1916us) server_negotiation.cc:407] Connection header received
0619 12:13:48.336807 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.336807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.336817 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.336848 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.337569 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.338310 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.339358 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.339652 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.339711 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.341861 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.341862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.341862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.341869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.344286 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.351948 (+ 7662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.351949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.351949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.352024 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.359337 (+ 7313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.359338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.359338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.359495 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.359586 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.360156 (+ 570us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.360247 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:13:48.361306 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.350656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.201.60:41879 (local address 127.0.0.1:48136)
0619 12:13:48.351095 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 12:13:48.351105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:13:48.351137 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:13:48.352495 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:13:48.352496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:13:48.352679 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:13:48.352718 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.352720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.354031 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.354033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.354876 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:13:48.354878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.355481 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.355482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:13:48.355558 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.355926 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:13:48.355930 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:13:48.357370 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:13:48.359812 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.359812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.359813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.360008 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.360302 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:13:48.360303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:13:48.360303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:13:48.360367 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:13:48.360998 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:13:48.360999 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:13:48.361054 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 12:13:48.361109 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 12:13:48.361210 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420}
I20260619 12:13:48.361929 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:48.351253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.0.201.60:41879)
0619 12:13:48.352324 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.352326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.352334 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:48.352369 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.352370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.352380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.352433 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.352821 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.353923 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.355044 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:48.355352 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:48.355416 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:48.357449 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:13:48.357451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:13:48.357451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:13:48.357458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:13:48.359694 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.360094 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.360095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.360096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.360177 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:13:48.360660 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:13:48.360661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:13:48.360661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:13:48.360866 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:13:48.361607 (+ 741us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:48.361698 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 12:13:48.361799 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049}
I20260619 12:13:48.363677 23519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: Bootstrap starting.
I20260619 12:13:48.364404 23519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: Neither blocks nor log segments found. Creating new log.
I20260619 12:13:48.364691 23519 log.cc:824] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:13:48.376091 23519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: No bootstrap required, opened a new log
I20260619 12:13:48.378230 23519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.378382 23519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4734fd64c3584548a01a5264df8059e5, State: Initialized, Role: FOLLOWER
I20260619 12:13:48.378473 23519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } }
I20260619 12:13:48.378836 23519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:13:48.378906 23492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:13:48.379987 23492 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [term 1 FOLLOWER]: Refusing update from remote peer b700c9d70f074661b42ce628e89a9724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:13:48.380851 23614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:13:48.380949 23654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 12:13:48.381213 23493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b700c9d70f074661b42ce628e89a9724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4734fd64c3584548a01a5264df8059e5"
W20260619 12:13:48.384101 23654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:13:48.384565 23643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.384689 23643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.384778 23643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b700c9d70f074661b42ce628e89a9724. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.384868 23643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.384938 23643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.385017 23643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:13:48.385771 23493 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b700c9d70f074661b42ce628e89a9724 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260619 12:13:48.386597 23657 mvcc.cc:205] Tried to move back new op lower bound from 7298544551257395200 to 7298544551240429568. Current Snapshot: MvccSnapshot[applied={T|T < 7298544551257395200}]
I20260619 12:13:48.389634 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b700c9d70f074661b42ce628e89a9724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700c9d70f074661b42ce628e89a9724" member_type: VOTER last_known_addr { host: "127.0.201.62" port: 42293 } } peers { permanent_uuid: "4734fd64c3584548a01a5264df8059e5" member_type: VOTER last_known_addr { host: "127.0.201.61" port: 43907 } } peers { permanent_uuid: "a6bc036599b24ed2b25d73dbfef7be36" member_type: VOTER last_known_addr { host: "127.0.201.60" port: 41879 } } }
I20260619 12:13:48.389762 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724 [sys.catalog]: This master's current role is: LEADER
I20260619 12:13:48.391253 23629 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:13:48.392154 23629 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:13:48.405582 23629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b700c9d70f074661b42ce628e89a9724: Generated new TSK 0
I20260619 12:13:48.405797 23629 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:13:48.432212 23598 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.429926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.0.201.60:41879)
0619 12:13:48.429972 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.429974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.431369 (+ 1395us) server_negotiation.cc:407] Connection header received
0619 12:13:48.431563 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.431565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.431576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.431608 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.432143 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48142: BlockingRecv error: recv got EOF from 127.0.0.1:48142 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 12:13:48.433172 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.430684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.0.201.61:43907)
0619 12:13:48.430743 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.430746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.431456 (+ 710us) server_negotiation.cc:407] Connection header received
0619 12:13:48.432711 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.432712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.432725 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.432755 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.433112 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57548: BlockingRecv error: recv got EOF from 127.0.0.1:57548 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 12:13:48.433772 23448 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.430142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.0.201.62:42293)
0619 12:13:48.430179 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.430181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.431267 (+ 1086us) server_negotiation.cc:407] Connection header received
0619 12:13:48.432599 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.432600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.432613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.432643 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.433705 (+ 1062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48838: BlockingRecv error: recv got EOF from 127.0.0.1:48838 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 12:13:48.436290 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.435549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57558 (local address 127.0.201.61:43907)
0619 12:13:48.435595 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.435597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.435786 (+ 189us) server_negotiation.cc:407] Connection header received
0619 12:13:48.435885 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.435886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.435901 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.435930 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.436203 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57558: BlockingRecv error: recv got EOF from 127.0.0.1:57558 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 12:13:48.439347 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.438571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.0.201.61:43907)
0619 12:13:48.438601 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.438603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.438842 (+ 239us) server_negotiation.cc:407] Connection header received
0619 12:13:48.438931 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.438932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.438941 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.438969 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.439305 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57568: BlockingRecv error: recv got EOF from 127.0.0.1:57568 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 12:13:48.443240 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.442517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.0.201.61:43907)
0619 12:13:48.442549 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.442551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.442763 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:13:48.442850 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.442851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.442860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.442888 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.443163 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57578: BlockingRecv error: recv got EOF from 127.0.0.1:57578 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 12:13:48.449136 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.448342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.0.201.61:43907)
0619 12:13:48.448377 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.448379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.448641 (+ 262us) server_negotiation.cc:407] Connection header received
0619 12:13:48.448736 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.448737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.448746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.448775 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.449089 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57582: BlockingRecv error: recv got EOF from 127.0.0.1:57582 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 12:13:48.459276 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.458398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57596 (local address 127.0.201.61:43907)
0619 12:13:48.458437 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.458439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.458706 (+ 267us) server_negotiation.cc:407] Connection header received
0619 12:13:48.458807 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.458808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.458819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.458849 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.459187 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57596: BlockingRecv error: recv got EOF from 127.0.0.1:57596 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 12:13:48.477686 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.476717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.0.201.61:43907)
0619 12:13:48.476764 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.476766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.477066 (+ 300us) server_negotiation.cc:407] Connection header received
0619 12:13:48.477177 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.477178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.477190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.477233 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.477625 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57612: BlockingRecv error: recv got EOF from 127.0.0.1:57612 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 12:13:48.511991 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.511162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57614 (local address 127.0.201.61:43907)
0619 12:13:48.511227 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.511230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.511373 (+ 143us) server_negotiation.cc:407] Connection header received
0619 12:13:48.511427 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.511429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.511442 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.511521 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.511933 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57614: BlockingRecv error: recv got EOF from 127.0.0.1:57614 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 12:13:48.578151 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.577437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57630 (local address 127.0.201.61:43907)
0619 12:13:48.577506 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.577509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.577615 (+ 106us) server_negotiation.cc:407] Connection header received
0619 12:13:48.577667 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.577668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.577678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.577742 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.578070 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57630: BlockingRecv error: recv got EOF from 127.0.0.1:57630 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 12:13:48.708035 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.707378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.0.201.61:43907)
0619 12:13:48.707441 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.707444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.707542 (+ 98us) server_negotiation.cc:407] Connection header received
0619 12:13:48.707595 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.707596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.707605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.707656 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.707977 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57634: BlockingRecv error: recv got EOF from 127.0.0.1:57634 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 12:13:48.966061 23524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:13:48.965412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.0.201.61:43907)
0619 12:13:48.965489 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:48.965491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:48.965502 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:13:48.965596 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:48.965597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:48.965614 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:48.965677 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:13:48.966011 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57648: BlockingRecv error: recv got EOF from 127.0.0.1:57648 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:13:49.299896 23611 catalog_manager.cc:1259] Loaded cluster ID: c34fd4f9f57e488b94a1c9c5feae0348
I20260619 12:13:49.299974 23611 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: loading cluster ID for follower catalog manager: success
I20260619 12:13:49.300737 23611 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: acquiring CA information for follower catalog manager: success
I20260619 12:13:49.300987 23611 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6bc036599b24ed2b25d73dbfef7be36: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:13:49.384606 23654 catalog_manager.cc:1259] Loaded cluster ID: c34fd4f9f57e488b94a1c9c5feae0348
I20260619 12:13:49.384702 23654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: loading cluster ID for follower catalog manager: success
I20260619 12:13:49.385509 23654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: acquiring CA information for follower catalog manager: success
I20260619 12:13:49.385816 23654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4734fd64c3584548a01a5264df8059e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:13:49.482158 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:49.479414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.0.201.61:43907)
0619 12:13:49.479536 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:49.479538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:49.479585 (+ 47us) server_negotiation.cc:407] Connection header received
0619 12:13:49.479703 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:49.479704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:49.479753 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:49.479837 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:49.479995 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.480569 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.481342 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.481553 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.481642 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:49.481812 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:49.481927 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:49.482012 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:49.482035 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 12:13:49.482054 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":51,"threads_started":1}
I20260619 12:13:49.486415 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:49.483931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.0.201.60:41879)
0619 12:13:49.484051 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:49.484053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:49.484062 (+ 9us) server_negotiation.cc:407] Connection header received
0619 12:13:49.484162 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:49.484163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:49.484213 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:49.484297 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:49.484413 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.484895 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.485634 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.485844 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.485924 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:49.486076 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:49.486181 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:49.486254 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:49.486307 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:13:49.486328 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":47,"threads_started":1}
I20260619 12:13:49.490818 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:49.488175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.0.201.62:42293)
0619 12:13:49.488301 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:49.488304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:49.488312 (+ 8us) server_negotiation.cc:407] Connection header received
0619 12:13:49.488358 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:49.488359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:49.488407 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:49.488480 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:49.488660 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.489201 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.489965 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.490179 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.490248 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:49.490444 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:49.490541 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:49.490631 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:49.490687 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 12:13:49.490710 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260619 12:13:49.491227 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:49.488066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57676 (local address 127.0.201.61:43907)
0619 12:13:49.488119 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:49.488122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:49.488328 (+ 206us) server_negotiation.cc:407] Connection header received
0619 12:13:49.488373 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:49.488373 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:49.488383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:49.488430 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:49.488564 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.489244 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.490360 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.490647 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.490704 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:49.490870 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:49.490906 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:49.490958 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:49.491113 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 12:13:49.491136 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:13:49.491524 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:13:49.488105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.0.201.60:41879)
0619 12:13:49.488155 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 12:13:49.488157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 12:13:49.488444 (+ 287us) server_negotiation.cc:407] Connection header received
0619 12:13:49.488497 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:13:49.488497 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:13:49.488508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:13:49.488561 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:13:49.488788 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.489477 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.490433 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:13:49.490723 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:13:49.490778 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:13:49.491055 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:13:49.491088 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:13:49.491119 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0619 12:13:49.491258 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 12:13:49.491431 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:13:49.492710 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 23376
I20260619 12:13:49.498039 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 23451
I20260619 12:13:49.502862 804 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq9ki8M/build/release/bin/kudu with pid 23526
2026-06-19T12:13:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2828 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5665 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (266343 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST